[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Another update on the Guix Data Service

From: Christopher Baines
Subject: Another update on the Guix Data Service
Date: Sun, 05 Jan 2020 20:15:37 +0000
User-agent: mu4e 1.2.0; emacs 26.3


It's been a while since I send out an update about the Guix Data
Service, so here's a somewhat failed attempt at a short update about
recent progress.

There are formatted [1] and plain views of derivations [2]. The plain
view is constructed from the representation in the database, but it
should match what Guix produces.


Channel news entries are recorded in the database, you can see all that
are present in a revision [3], but the comparison page should show new
news entries as well.


There's now a page to compare two derivations [4], it's not a
particularly clear comparison yet, but can hopefully be improved in the
future. This is linked to from the derivation history pages [5].


There's support for receiving build information from build servers, but
this hasn't been used yet. I'm slowly looking at getting support for
using this in to Cuirass [6]


Information about nars (substitutes) can now be stored ([7] for
example). This data also shows up on a page for derivation outputs for a
revision ([8] for example).


There's a page for builds for a revision [9]. There's also a page for
individual builds [10].


The derivation outputs page for a revision shows information about
outputs matching across build servers, but there's also a page about
package reproducibility for a revision [11].


Provide narinfo and nar files for derivations, and the required source
files ([12] for example). This means the Guix Data Service can be used
as a substitute server for derivations, for example [13].

13: guix build --substitute-urls=""; 

Fix generating derivations for foreign architectures. There was an issue
where only the x86_64-linux and i686-linux derivations were properly
loaded, but now that's been fixed.

I added support for loading mbox files a while ago, but I've only now
got around to testing a few older revisions, and now I've loaded the
mbox files for the first 3 months from 2019 so eventually those will be
loaded in.

A few things above were influenced by thinking about reproducible builds
in the run up, during and after the reproducible builds summit at the
start of December. I'm excited that there's some support in the Guix
Data Service for comparing builds across build farms, but there's still
a lot to do in terms of getting every derivation built by at least 2
separate build farms, and then getting that data promptly in to the Guix
Data Service for comparison.

Also at the reproducible builds summit, Ludovic made me aware that you
can tell Guix to build a derivation, and if it doesn't have it locally,
it will attempt to download it from substitute servers. I've now got the
Guix Data Service setup to act as a substitute server for derivations,
which while on it's own isn't particularly exciting (as build farms
already can provide substitutes for derivations), it's now possible to
get the information about derivations within the Guix Data Service, out
of the Guix Data Service in a usable manor. This might have applications
in the future, like speeding up guix pull by providing the derivation
that corresponds to a revision, or providing derivations for a build
farm to build.

Like the example of providing substitutes for derivations, quite a lot
of this work isn't directly impactful (at least yet), but I'm hoping
it's laid the foundations for some more directly impactful stuff.

In particular, I'm still thinking about;

 - Automated code review for Guix patches, which I was working on when I
   started working on the Guix Data Service, and most of the
   functionality can be beneficial there.

 - The Guix Weekly News project I was working on recently uses data from
   the Guix Data Service, and that's what motivated storing the channel
   news entries.

 - Trust in builds, once the data about package build reproducibility is
   more complete, hopefully that will be informative and show where
   build reproducibility issues can be fixed.

 - Improving the speed in which substitutes are made available.

 - Explaining the data that underpins Guix to a technical audience, like
   derivations and how they work.

Do let me know if you have any thoughts or questions, I'm also going to
be around at the Guix Days at the end of January.



Attachment: signature.asc
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]