Re: Build reproducibility metrics

[Ludovic Courtès]

Hello!

Christopher Baines <mail@cbaines.net> skribis:

> I've also been writing and trying to use the Guix Build Coordinator [1]
> to build Guix packages and provide substitutes. That has got to the
> point where it's not getting stuck every day at least, and there's more
> than 80% of packages are available.
>
> 1: https://git.cbaines.net/guix/build-coordinator/about/

Well done!

> Combining that with the substitute server operated by Tobias, which has
> a pretty awesome substitute availability of over 90% for recent
> revisions, not only is there data from 4 different substitute servers to
> use in the comparison, but the proportion of packages where there isn't
> sufficient data is pretty low, below 10%.
>
> I'm currently using the data.guix-patches.cbaines.net instance of the
> Guix Data Service, you can see the package substitute availability for
> the latest revision using this URL [1], and the package reproducibility
> at this URL [2].
>
> 1: 
> https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/package-substitute-availability
> 2: 
> https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/package-reproducibility

That’s really good!  It’s the first time we have this good an overview
of the package reproducibility status.

> Some caution is needed when interpreting this data. It's most probably
> less up to date than what you'd get through running the guix weather or
> guix challenge commands, as it takes the Guix Data Service time to query
> the data, that querying process isn't very reliable at the moment
> either. Additionally, the "matching" percentage could easily go down if
> that output is built with a different hash in the future.
>
> While the number itself maybe isn't the most useful thing, I like that
> clicking through to the "Not matching" outputs will show a list of
> outputs which didn't build reproducibly, which is something that could
> help identify reproducibility issues to investigate and fix.

Yes, definitely.  There’s also always the option of running ‘guix
challenge’ locally.

> I think things are coming together on the substitute server side. The
> goal I have in mind for this is for users of Guix to be able to have
> greater trust in the substitutes they use, through trusting substitutes
> only if it's been built reproducibly on multiple substitute servers. It
> would be great to see work start soon on how guix as a client to
> substitute servers might be enhanced to check for reproducibility when
> fetching substitutes.

Agreed!

I think between that, the reduced bootstrap seeds, and authenticated
checkouts, we’re starting to have a good security story.

Thank you!

Ludo’.