Re: Introducing Guix to HPC at my institution

[Sébastien Lerique]

Hi Ludo,

Thanks for the help!

On 29 Mar 2021 at 21:03, Ludovic Courtès 
<ludovic.courtes@inria.fr> wrote:

> Instead of installing the “regular” binary tarball inside a 
> namespace,
> it might be easier to create a tarball like so:
>
>   guix pack -RR -S /bin=bin -S /etc=etc guix bash
>
> … and to unpack the resulting tarball.
>
> From there, you can run ./bin/sh to get a shell that “sees” 
> /gnu/store.
> You can then run:
>
>   . ./etc/profile
>
> And then, you should be able to run the daemon, like so:
>
>   export GUIX_STATE_DIRECTORY=$HOME/.local/var/guix
>   guix-daemon --disable-chroot &
>
> (Adapted from
> <https://lists.gnu.org/archive/html/guix-devel/2018-05/msg00139.html>.)
>
> Does that work for you?

Thanks! I `guix pack`'ed with a single -R since I shouldn't need 
PRoot, and had to add

 export GUIX_LOG_DIRECTORY=$HOME/.local/var/log

to the environment setup in order to start guix-daemon. Then `guix 
install hello` warned at the beginning with

 user with UID 7352 not found

and later failed with

 Backtrace:
 In ice-9/boot-9.scm:
   1736:10  6 (with-exception-handler _ _ #:unwind? _ # _)
 In unknown file:
            5 (apply-smob/0 #<thunk 7f5a32f85520>)
 In ice-9/boot-9.scm:
     718:2  4 (call-with-prompt _ _ #<procedure 
     default-prompt-handle…>)
 In ice-9/eval.scm:
     619:8  3 (_ #(#(#<directory (guile-user) 7f5a32f88c80>)))
 In guix/ui.scm:
   2164:12  2 (run-guix-command _ . _)
 In guix/scripts/offload.scm:
    782:21  1 (guix-offload "x86_64-linux" "0" "1" "0")
 In unknown file:
            0 (getpw 7352)

 ERROR: In procedure getpw:
 In procedure getpw: entry not found
 killing process 9106
 guix install: error: unexpected EOF reading a line

I'm guessing some part on the non-namespaced uids are leaking into 
the user namespace?


Another problem appeared for substitutes (as 'guix install hello' 
was rebuilding the world):

 $ guix archive --authorize < \
   
/gnu/store/xb4szjambyi52bpnkjv080g2mlfqqpp0-profile/share/guix/ci.guix.gnu.org.pub
 guix archive: error: mkdir: Permission denied


I quickly went through `guix/scripts/pack.scm` and 
`gnu/packages/aux-files/run-in-namespace.c` to try and understand 
what `guix pack -R` does, but it looks like I need to learn more 
about namespaces before I can digest that. My high-level 
understanding from poking around is that the binaries from `guix 
pack -R` run inside a namespace where /gnu is bind-mounted (along 
with /proc, /dev, /sys, ...), but the rest of the host filesystem 
is still available, and other host processes can also be seen. 
Then, anything I run from inside the `guix pack`'ed bash inherits 
that namespace, which is what makes guix-daemon work.

If this is incorrect, could you maybe give a high-level overview 
of what it does? The documentation is a bit scarce on the topic 
(or I didn't find it).

I'm also wondering why `guix-daemon` must be invoked with 
`--disable-chroot`. Doesn't that make the reproducibility 
guarantees more brittle (according to the docs)?

> > - is it possible to create build users inside the 
> > user-namespaced
> >   chroot?
>
> No: you still have a single UID at hand, so there’s no way to 
> allocate
> new ones.
>
> > - last but not least, how would I go about sharing this setup 
> > with
> >   other users on the cluster? Ideally I would like to have a
> >  non-priviliged build daemon that other users can call on. (Is 
> >  there
> > such a thing as kernel group namespaces?)
>
> It’s not really sharable.  To share it, you would need some sort 
> of a
> shared trusted “proxy”; that’s precisely what guix-daemon is in 
> normal
> multi-user setups.

Ok. My use cases here are:
- trying to provide a setup that others can more-or-less easily 
 try out to get a feel for Guix on the HPC cluster (if possible 
 without duplicating /gnu/store), thus creating interest and user 
 support for later presenting the case to sysadmins / university 
 administration;
- getting to know the exact requirements that I would have to 
 convince sysadmins to agree to, with rationales/explanations for 
 the hardest-to-obtain. Getting them to run guix-daemon as root 
 is probably a lost case, so I'd like to explore everything else 
 that is possible.

So let's transform my two questions above into the two following:

1. what would be the list of requirements that explain why 
guix-daemon must run as root? What does each of those requirements 
accomplish?
2. knowing that my environment has user namespaces (and knowing 
that that may not be enough), would it be possible to set up 
guix-daemon as non-root with the assistance of a sysadmin (so say 
with root access during setup), with build users and all, and have 
it provide identical guarantees to a guix-daemon running as root? 
If not, why not, and could we work around that?


Thanks again for all the help!
Sébastien