[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Guix on clusters and in HPC
From: |
Ludovic Courtès |
Subject: |
Re: Guix on clusters and in HPC |
Date: |
Thu, 20 Oct 2016 16:08:29 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) |
Hi Roel,
Roel Janssen <address@hidden> skribis:
> Here are some aspects I think we need:
>
> * Network-aware guix-daemon
Of course!
> * Profile management
>
> The abstraction of profiles is an awesome feature of FPM, but the user
> interface is missing. We could do better here.
>
> Switch the default profile
> (and prepend values of environment variables to the current values):
> $ guix profile --switch=/path/to/shared/profile
>
> Reset to default profile (and environment variable values without the
> profile we just unset):
> $ guix profile --reset
>
> Create an isolated environment based on a profile:
> $ guix environment --profile=/path/to/profile --pure --ad-hoc
I can see the desire of having something that more closely resembles
what “modules” does, but I have the same questions as Ricardo.
Essentially, how much would it help compared to what’s already
available? (Honest question.)
In general, adding simpler UIs is a good idea IMO; it’s just that I’m
unsure what’s “better” in this case.
> * Workflow management/execution
>
> Add automatic program execution with its own vocabulary. I think
> "workflow management" boils down to execution of a G-exp, but the
> results do not necessarily need to be stored in the store (because the
> data it works on is probably managed by an external data management
> system). A powerful feature of GNU Guix is its domain-specific
> language for describing software packages. We could add
> domain-specific parts for workflow management (a `workflow' data type
> and a `task' or `process' data type gets us there more or less).
>
> With workflow management we are only interested in the "build
> function", not the "source code" or the "build output".
>
> You are probably aware that I worked on this for some time, so I could
> share the data types I have and the execution engine parts I have.
Yes, definitely! This is what I had in mind, hence the reference to
<https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00380.html>.
Obviously if there’s already code, it’s even better. :-)
> The HPC-specific part of this is the compatibility with existing job
> scheduling systems and data management systems.
Do you mean that it integrates with a job scheduler?
> * Document on why we need super user privileges on the Guix daemon
>
> Probably an infamous point by now. By design, the Linux kernel keeps
> control over all processes. With GNU Guix, we need some control over
> the environment in which a process runs (disable network access,
> change the user that executes a process), and the environment in which
> the output lives (chown, chmod, to allow multiple users to use the
> build output). Instead of hitting the wall of "we are not going to
> run this thing with root privileges", we could present our sysadmins
> with a document for the reasons, the design decisions and the actual
> code involved in super user privilege stuff.
>
> This is something I am working on as well, but help is always welcome
> :-).
Good point.
<https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html>
mentions it when talking about --disable-chroot at the end, but this
could be improved.
That’s it? No crazy ideas? ;-)
Your thoughts about the point about Galaxy?
Thanks for your feedback!
Ludo’.