Re: Installing guix packages without root permissions (in HPC environments)

[Ludovic Courtès]

Hi Pjotr!

Pjotr Prins <address@hidden> skribis:

> But, I thought the easy way is to patch a path with something the has
> the exact same size(!). This has the advantage that it will always
> work. Trying this second strategy I wrote a new tool which replaces
> the old path with a new one that takes the prefix and truncates the
> rest of the path so a prefix /usr/local/bin/hello overwrites
>
> Found @512:     /gnu/store/qv7bk62c22ms9i11dhfl71hnivyc82k2-glibc-2.22/...
> Replace with    /usr/local/bin/hello/glibc-2.22-qv7bk62c22ms9i11dhfl71/...
>
> You can see we swap the hash position and start 'eating' the path from
> the end.

Ah ha, clever and fun!  :-)

> This should work across almost all files, unless the path is scrambled
> in some way. The downside of the fixed strategy is that a prefix can
> not grow beyond the size of the one in the store. Also every store
> path may look a bit different between installs.

The latter is the price to pay, I guess.

> You can try it out by downloading and unpacking
>
>     http://biogems.info/contrib/genenetwork/guix-build-hello-2.10-x86_64.tgz
>
> Run the contained installer with
>
>     ./install.sh prefix
>
> Use --help, -v and -d for more output.  And run the installed tool
> with
>
>     env LC_ALL=fr_FR prefix/profile-hfmsjsvx1p68wbx0fli/bin/hello
>     Bonjour, le monde!
>
> French (if the locale resolves ;).  On non-guix machines I have
> already deployed some complex packages this way, including Ruby,
> sambamba and the ldc compiler (which includes LLVM).

Woow, good to know that it works well in practice.

> To create such an installable and relocatable tarball start from a
> package with its dependencies, such as found in a Guix archive or
> created with
>
>     guix environment --container --ad-hoc mypackage tar gzip -- tar cvzf 
> mypackage.tgz /gnu/store
>
> Unpack that tar ball and copy the installer files that sit in the
> hello example. That is all.
>
> The source code for the guix-relocator is at
> https://github.com/pjotrp/guix-relocate/blob/master/src/main.d and the
> supporting scripts are at https://github.com/pjotrp/gnu-install-bin (I
> may have to change that name).
>
> To fix the prefix restriction there are two routes, one is building
> Guix itself on a large(r) path, which is the easy route, or write a
> relocate patcher that can handle the non-zero terminated paths. I
> think the latter is feasible too. Even at this point the fixed length
> strategy is useful for most environments - the prefix can be some 40
> characters long.

I think the fixed-length restriction is not that bad.  The two
workarounds above look hacky and super tricky, respectively.  ;-)

Also, if we look at the big picture of non-root usage, this solution
addresses the most hostile environments: no user namespaces, no
container-spawning facility, no root guix-daemon, etc.  Granted, these
hostile environments are still commonplace in HPC, so that’s good.  :-)

> What does this all mean? In short, if the prefix is not too long we
> can run practically all Guix binary software on a non-guix host with
> normal user permissions. Something we need for HPC and supercomputing
> and is the subject of my FOSDEM talk on the HPC track:
>
>   https://fosdem.org/2017/schedule/event/hpc_deployment_guix/
>
> There is also scope for creating a one-step installer. A (trusted)
> server could distribute binary software with its dependencies to any
> Linux machine. I am sure software developers are interested in that.

Awesome!

Ludo’.