Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <address@hidden> skribis:

> Sure, I can take care of updating NEWS in the next day or two.

Awesome, thank you!

>>> Regrettably, Guile 2.2 has become too heavy to build on the only machine
>>> in my possession that I have any trust in.  I don't have a machine that
>>> I consider sufficiently trustworthy to produce build outputs for wider
>>> distribution.  I'm not sure that any of us do.
>>
>> Note that “make dist” is rather inexpensive;
>
> I assume it builds the prebuilt .go files, no?  That involves running
> Guile's compiler, which is too heavy to run on my Yeeloong.

I think it’s not that bad if you already have build tree with Guile’s
compiler.  If you start from scratch, it’s definitely expensive.

> If you'd like to produce the 2.2.5 release in the traditional way,
> that's fine with me.  I'm not comfortable doing it myself, though.

OK, I can do this.

>> One issue is that “make dist” is non-deterministic because the archive
>> contains timestamps; I’m sure there of other sources of non-determinism
>> though, because “make dist” was not designed with that in mind.
>
> Right.  I suppose the right approach is to start a conversation with the
> autotools developers.  In the meantime, I wonder if we could implement
> our own deterministic version of "make dist" using Guix, and use that
> instead.  Or perhaps it would be easier to use "make dist" and then
> canonicalize the timestamps in the resulting tarball in a later step?
>
> Thoughts?

I think you raise valid concerns, but they are to some extent beyond the
scope of Guile.

Regarding “make dist”, there’s the issue of tar timestamps, of
version.texi, and probably others of that sort in the
autotools-generated machinery.

So yes, I think this should be discussed with the Automake/Autotools
developers.  Namely: how can we achieve reproducible “dist” builds?
Which tools should honor SOURCE_DATE_EPOCH and how? etc.

As a PoC and/or interim solution, we could also try to hack a
reproducible “make dist” in Guix.

For Guix, from a bootstrapping viewpoint, the alternative is to do away
with tarballs produced by “make dist” and instead always run
“autoreconf” ourselves, like Debian does.  That would solve a large part
of the problem.

>> The non-source byproducts in release tarballs are: the pre-built .go
>> files (which are optional), psyntax-pp.scm, and then Info files and all
>> the autotools machinery.  Are these those you had in mind?
>
> Yes, all of the above are potential security risks, except possibly for
> the info files.

I think psyntax-pp.scm is the main issue since all the others can be
trivially rebuilt (the package in Guix deletes the pre-built .go files
for instance.)

With regards to bootstrapping in the context of Guile, I believe
psyntax-pp.scm should be our primary concern.

Thanks,
Ludo’.