[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Merging staging?
Mark H Weaver
Re: Merging staging?
Thu, 20 Dec 2018 20:17:07 -0500
Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
I've rearranged your reply from "top-posting" style to "bottom-posting"
style. Please consider using bottom-posting in the future.
> Julien Lepiller <address@hidden> writes:
>> I'd like to get staging merged soon, as it wasn't for quite some
>> time. Here are some stats about the current state of substitutes for
>> According to guix weather, we have:
>> | architecture | berlin | hydra |
>> | x86_64 | 36.5% | 81.7% |
>> | i686 | 23.8% | 71.0% |
>> | aarch64 | 22.2% | 00.0% |
>> | armhf | 17.0% | 45.6% |
>> What should the next step be?
> I think we should wait until the coverage on armhf and aarch64 have
> become larger, for the sake of users on those systems.
> Also, I've seen some commits that make me wonder if hydra is still
> being configured as an authorized substitute server on new Guix
> Do you know?
> If 'berlin' is the only substitute server by default, then we certainly
> need to wait for those numbers to get higher, no?
> What do you think?
Julien Lepiller <address@hidden> responded:
> I agree, but I wonder if there is a reason for these to be so low?
It's a good question. I have several hypotheses:
* Unfortunately, it is fairly common for builds for important core
packages to spuriously fail, often due to unreliable test suites, and
to cause thousands of other important dependent packages to fail.
When this happens on Hydra, I can see what's going on, and restart the
build and all of its dependents.
I wouldn't be surprised if some important core packages spuriously
failed to build on Berlin, but we have no effective way to see what
happened there. If that's the case, the 'guix weather' numbers above
might never get much higher no matter how long we wait.
* Berlin's build slots may have been occupied for long periods of time
by 'test.*' jobs stuck in an endless "waiting for udevd..." loop, as
described in <https://bugs.gnu.org/33362>.
Hydra's web interface allows me to monitor active jobs and manually
kill those stuck jobs when I find them. I don't know how to do that
* Especially on armhf and aarch64, where Berlin has very little build
capacity, and new builds are being added to Berlin's build queue must
faster than they can be built, it is quite possible that Berlin is
spending most of its effort on long-outdated builds.
On Hydra, I can see when this is happening, and often intervene by
cancelling large numbers of outdated builds on armhf, so that it
remains focused on the most popular and up-to-date packages.
* On WIP branches like 'core-updates' and 'staging', when a new
evaluation is done, I cancel all outdated Hydra jobs on those
branches. I don't know if anything similar is done on Berlin.
In summary, there are several things that I regularly do to make
efficient use of Hydra's limited build capacity. I periodically look at
Berlin's web interface to see how it has progressed, but it is currently
mostly a black box to me. I see no effective way to focus its limited
resources on the most important builds, or to see when build slots are