guix-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Parallel guix builds can trample?


From: Phil Beadling
Subject: Re: Parallel guix builds can trample?
Date: Tue, 18 Jan 2022 12:53:48 +0000

Hi all,

The issue also exists when using --with-commit - see below for a refined test that makes it trivial to demonstrate with any package where the source is retrieved from git.

On Tue, 18 Jan 2022 at 10:10, Phil <phil@beadling.co.uk> wrote:

> Philip Beadling <phil@beadling.co.uk> writes:
 
>No, and we probably should do, even in spite of this issue.  We're going
>to repeat the experiment now using --with-commit, to confirm behaviour there.


Let's try and build 2 package variations - for this experiement we don't even have to run them at the same time:

mkdir checkout-test
cd checkout-test
mkdir cache1 cache2
XDG_CACHE_HOME=`realpath ./cache1` guix build --with-commit=simm=3981f60d81d3c9592d78b3e88545259713a93194 simm
XDG_CACHE_HOME=`realpath ./cache2` guix build --with-commit=simm=ec1357c1fdf78a2ac1408d00c0c3e7d025ab9ade simm

Now look at the caches:

$ ll cache1/guix/checkouts

total 4.0K
-rw-rw-r-- 1 phil.beadling phil.beadling  10 Jan 18 12:37 last-expiry-cleanup
drwxrwxr-x 6 phil.beadling phil.beadling 162 Jan 18 12:37 lx4ipn27sk6uvu6b33kelt4tik65oianvl7b7yn5bpng7gpt4pyq
drwxrwxr-x 5 phil.beadling phil.beadling 115 Jan 18 12:37 mqqlbc4ksftu2tkbnxydosm4j6mynxsh5nmhpq3cfrofkktkhexq
drwxrwxr-x 5 phil.beadling phil.beadling 103 Jan 18 12:37 qqzuuoqxvqikivkpenqom43kzk2xr7nlntanbm6fpe7bci6kaenq
drwxrwxr-x 4 phil.beadling phil.beadling 202 Jan 18 12:37 smvbhjf46cbaxxju6fsqj7t6ujktxy5i7vhuhzpvgjt2vhid2vqq
drwxrwxr-x 5 phil.beadling phil.beadling 267 Jan 18 12:37 xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq

$ ll cache2/guix/checkouts

total 4.0K
-rw-rw-r-- 1 phil.beadling phil.beadling  10 Jan 18 12:38 last-expiry-cleanup
drwxrwxr-x 6 phil.beadling phil.beadling 162 Jan 18 12:38 lx4ipn27sk6uvu6b33kelt4tik65oianvl7b7yn5bpng7gpt4pyq
drwxrwxr-x 5 phil.beadling phil.beadling 115 Jan 18 12:38 mqqlbc4ksftu2tkbnxydosm4j6mynxsh5nmhpq3cfrofkktkhexq
drwxrwxr-x 5 phil.beadling phil.beadling 103 Jan 18 12:39 qqzuuoqxvqikivkpenqom43kzk2xr7nlntanbm6fpe7bci6kaenq
drwxrwxr-x 4 phil.beadling phil.beadling 202 Jan 18 12:38 smvbhjf46cbaxxju6fsqj7t6ujktxy5i7vhuhzpvgjt2vhid2vqq
drwxrwxr-x 5 phil.beadling phil.beadling 267 Jan 18 12:38 xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq

$

Despite asking for different variants of the simm package, they are both stored under the same SHA.  If they both used the same cache directory (the default) - they'd have overwritten each other.  Thus it is not safe to run --with-commit or any other variation in parallel for the same user.

To prove I've matched the right SHA - I can back-out the original commit ids provided to --with-commit:

$ cd cache1/guix/checkouts/xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq && git rev-parse HEAD && cd -

3981f60d81d3c9592d78b3e88545259713a93194
~/checkout-test

$ cd cache2/guix/checkouts/xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq && git rev-parse HEAD && cd -

ec1357c1fdf78a2ac1408d00c0c3e7d025ab9ade
~/checkout-test

$

Whilst my original setup was a bit of a edge-case, I think this refined example demonstrates that the problem can occur with typical/reasonable use of guix switches?  I can even think of how this could occur without the user running in parallel - for example if a package required via nth-order dependencies to build 2 versions of the same repo - it's possible that a single "guix build" command could cause the issue?

Thoughts?



reply via email to

[Prev in Thread] Current Thread [Next in Thread]