bug#73602: bug#73601: failed to compute the derivation for Guix

[Simon Tournier]

Hi Fabrice,

On Fri, 11 Oct 2024 at 15:13, Fabrice Tudoret <fabrice.tudoret@univ-rennes1.fr> 
wrote:

> GUIX seem's to work fine with the root account and the local users but
> not with ldap users.

Ah, that doesn’t ring a bell but maybe it’s related.  I don’t know.


> 1-

Just to be sure and since you have reinstalled, what is your Guix
revision?

On a side note, since it seems fine for the root account and here you
run it as root, I guess all is fine. :-)  Although I don’t have the
same… 🤔

> [root@cluster24 ~]# cat 
> /gnu/store/jc3vgcsplqsim3na80b0n2iilna5j6gx-Python-3.5.9.tar.xz.drv  | sed 
> 's/)/)\n/g'
> Derive([("out","/gnu/store/cv4h89n30myf3nhjqnnahlbij2gaw21z-Python-3.5.9.tar.xz","","")
> ],[("/gnu/store/21c7pjahkh20mmzq2ivki57zwwvp6nwn-bootstrap-binaries-0.drv",["out"])
> ,("/gnu/store/5gf7f8awndhnf2gn2mzbfbqr3ix9aj80-module-import-compiled.drv",["out"])
> ,("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])
> ,("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])
> ],["/gnu/store/1s8jdafkyhz0p81l0j37yih9gbrb5gix-module-import","/gnu/store/h58cvdcdak4d87lw0fkvmkhan95ssljx-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/1s8jdafkyhz0p81l0j37yih9gbrb5gix-module-import","-C","/gnu/store/gz5rcilhcsc5amgxcgyxvn0s5px8sg80-module-import-compiled","/gnu/store/h58cvdcdak4d87lw0fkvmkhan95ssljx-Python-3.5.9.tar.xz-builder"],[("guix
>  properties","((type . origin)
>  (patches . 0)
> )
> ")
> ,("out","/gnu/store/cv4h89n30myf3nhjqnnahlbij2gaw21z-Python-3.5.9.tar.xz")
> ])
>
>
> 2 -The script builder
> /gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder
> is well present in /gnu/store.

Ouf. :-)


Well, since it works as expected when run as root (#) and it fails when
run as a regular ($), I propose to diff various files in order to spot
what could be wrong.

Some details about some internals – well my understanding and I’m
perhaps missing important points –, then maybe they will explain the
logic behind the exploration. :-)

Roughly speaking, the items in the store look like:

    /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-foobar-1.2.3

where ’foobar-1.2.3’ is a “label” corresponding to the package and
’xxx…’ is some hash.  This hash is the core of the content-addressed;
the one that allow the substitution, i.e., download the artefacts.

Basically, this hash is computed by hashing the inputs and the script
builders.  Therefore somehow it builds a chain and the roots are named
fixed-outputs.  Fixed-outputs are items for which we known beforehand
the resulting hash.  Else we cannot know the hash beforehand because
it’s hard to know beforehand the checksum of the artefact since the
artefact is the result of the build process (compilation, etc.).

In other words, the expectation is: the same inputs and the same builder
script returns the same store item.  And the derivation captures that.
Well, for sure the content of the store item on two machines is the same
only if the process is fully deterministic; another story. ;-)

All that to say: if we scrutinize the derivations and the builder
scripts, then we will spot what introduces a difference.

Aside, please note that two different derivations might produce the same
store item, see for example [1].

Let’s go! :-)


a) Both root and regular must use the exact same Guix revision.

  # As root
  # readlink -f (type -P guix)
  # guix describe

  $ As regular user
  $ readlink -f (type -P guix)
  $ guix describe


b) If that’s not the case, it isn’t an issue for the following but it’s
   better to know. :-)


c) Display the both derivations, as root and as regular user

  # guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d

  $ guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d


The expected situation is: as root,

  # guix build -e '(@@ (gnu packages commencement) python-boot0)' -S

it just downloads stuff but as regular user:

  $ guix build -e '(@@ (gnu packages commencement) python-boot0)' -S

then it builds stuff and one build fails.

Correct?


d) Display the content of the both derivations, as root and and regular
   user.

  drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
  cat $drv | sed 's/),/\n),/g'

Now we can try to find what can be wrong. :-)


Roughly, the inputs are one fixed-output and these three others:

    /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv
    /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv
    /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv

Well, please note that the hash of these derivations might vary but not
their output.  For instance,

    # sed 's/]/\n]/g' \
       /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv \   
      | head -1
    
Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")

Here the hash ’3895…’ might be different but not ’az8fi8…’.  The same
derivation produces the same store item though.

Ok, let consider each.  I know that piping with “sed” is not handy and
an extension [2] ]is still pending… Anyway, please pipe the derivations
in order to extract the relevant information.


1. /gnu/store/…-guile-bootstrap: it only depends on fixed-outputs so the
only source of variation comes from the script builder: in my case,

    /gnu/store/nmxl6qyj16bv4rx4irhg23r66gn752kd-build-bootstrap-guile.sh

I expect you have the same.  As root, you have because it’s the same
derivation, namely
/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv.

What about the regular user?


2. /gnu/store/…-bootstrap-binaries: the inputs reads,

    /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
    /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv
    /gnu/store/nf9h1l2apgjq1gjdv1ynpgqrmaj1bawb-static-binaries.tar.xz.drv
    /gnu/store/sik0rp63q7hz4wry27v3ynljdmqvg194-xz.drv
    /gnu/store/zn60yim3km5qx8d0vzvf9izfxzfdy4x0-tar.drv

Assuming #1 (identical guile-bootstrap) and the only non fixed-output is
module-import-compiled.  Or the script builder.

Do you have the same script builder, namely

  /gnu/store/kv2vjnp7f75wnq8rp579ihz96knqp0sl-bootstrap-binaries-0-builder

?  I guess you have the same.  Therefore, I guess the issue is from
module-import-compiled.


3. /gnu/store/…-module-import-compiled: On my machine, it reads:

--8<---------------cut here---------------start------------->8---
Derive
([("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","","")]
 
,[("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])]
 
,["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/l36kgiw25r2a4vllwad18jg8jnkvmzda-build-utils.scm","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
 
,"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
 ,[("GUILE_WARN_DEPRECATED","no")
   ,("extensions","")
   ,("module count","1")
   ,("modules","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import")
   ,("optimization level","1")
   ,("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled")
   ,("preferLocalBuild","1")])
--8<---------------cut here---------------end--------------->8---

I guess you also have the same build-utils.scm.  Don’t you?

Therefore, I guess the difference is either an environment variable (the
list starting with GUILE_WARN_DEPRECATED) or the file
/gnu/store/…-qcompile-modules.  Hum, it would be weird that it comes
from compile-modules…


Ah my train is at destination.  To be continued… Let me know!

Cheers,
simon


1: Re: Derivations differ between computers?
zimoun <zimon.toutoune@gmail.com>
Fri, 26 Nov 2021 01:49:00 +0100
id:86lf1bspvn.fsf@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2021-11
https://yhetil.org/guix/86lf1bspvn.fsf@gmail.cmo

2: Guix extension to display derivation (and rewrite fixed-output)
Simon Tournier <zimon.toutoune@gmail.com>
Fri, 12 Apr 2024 20:28:11 +0200
id:87ttk6phac.fsf@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2024-04
https://yhetil.org/guix/87ttk6phac.fsf@gmail.com