[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: plz is there a roadmap for a more resilient substitutes infrastructu

From: Ludovic Courtès
Subject: Re: plz is there a roadmap for a more resilient substitutes infrastructure?
Date: Tue, 06 Nov 2018 12:23:05 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)

Ciao Giovanni,

Giovanni Biscuolo <address@hidden> skribis:

> recently users and developers are facing hard to manage problems due to
> the maintainance of and its proxy [1]
> since 23 Oct 2018

We Guix developers don’t have control over the physical hardware behind; for this machine, we rely on the work of the FSF
sysadmins for all things hardware/networking.

Unfortunately in this case, this maintenance period was rather
unprepared: it wasn’t supposed to last a whole week, rather a few hours
or a day at most.  Most of the time it took was about copying data to a
new disk (!).  Had this been prepared, we could have arranged to keep up until the replacement was ready.  We Guix developers
didn’t have much visibility over what was going on though, and we just
didn’t anticipate this.

It is clear that this prolonged downtime was harmful to many users and
to the project’s reputation.

What to do from here?

Our main focus is on making the primary build farm of
the project.  It has the advantage that one Guix dev has physical access
to it (Ricardo); it’s also much more powerful than and the
associated build machines.

Yet, there’s more work to do: berlin has just 1T of disk space.  Ricardo
started looking on growing it but was stuck on software issues IIRC.  I
think fixing this should be a priority, so I think we should help
Ricardo fix the software issues as much as we can.

That alone doesn’t fix the resilience issue: could go
down at some point for some time.

To address that, a possibility that was discussed recently on
guix-sysadmin is use has a separate build farm
and/or mirror of berlin.

On top of that we could have a service like, or
maybe even a CDN where we’d replicate substitutes, or torrents (looking
at you, Julien ;-)).

At this point, all these options are still on the table, and anyone with
expertise in these areas is very much welcome!

> given the prolonged issue, please also consider writing an *official*
> blog post explaining the current situation and steps adopted to prevent
> similar issues in the future

We set up the info-guix mailing list with that in mind (but too late for
this incident).  Posting blog posts is also a good idea; we should have
done that, with instructions on how to switch to

> 1. is there a method to "replicate the whole store of an official server
> (e.g. once healed)" so we can just "guix publish" a
> *complete* mirror? In this case a ready to use official
> mirror-config.scm could be useful is a simple nginx proxy to  You can
find its config here:

In the past a few people set up their own mirrors using a similar

> 2. is there an official mirrors directory users can look at when needed?


> 3. is there a plan to build a service similar to
> (I looked on the web but did not find any
> reference to such plan)

Like I wrote, there’s no concrete plan at this point, which means it’s
an opportunity for you and anyone else to chime in and give a hand!


reply via email to

[Prev in Thread] Current Thread [Next in Thread]