Re: Introducing Guix to HPC at my institution

[Sébastien Lerique]

Hi Ludo, Simon,

Thanks for these very interesting stories! That's a lot of 
resources too, I will be watching some of the talks you linked to 
over the coming days. I'm currently contacting people who might 
already feel a strong need for what Guix can offer, to see what 
would be the best way to introduce the topic to the sysadmins. Who 
knows, maybe my institution can also support the development of 
Guix in some way :)

Will be back once I have some progress or more questions.

Best,
Sébastien

On 16 Mar 2021 at 18:05, Ludovic Courtès 
<ludovic.courtes@inria.fr> wrote:

> Hi Sébastien,
>
> Sébastien Lerique <sl@eauchat.org> skribis:
>
> > Another thought: given Ludo's position, I am imagining that 
> > Inria
> > Bordeaux runs Guix on some of their infrastructure, is that the
> > case? If so, can you share any details about how that came to 
> > happen?
>
> I’ve been working with HPC people at Inria and back then I was 
> working
> on Guix in my spare time.  The HPC folks had been using 
> “environment
> modules” forever but were looking for a solution that would 
> provide
> automation, primarily, and optionally some kind of 
> reproducibility.
>
> They invested into Spack first (actually based on my 
> recommendation; at
> the time I wasn’t sure Guix could meet all their requirements) 
> and grew
> dissatisfied after a couple of years, in particular because
> reproducibility became a more important factor for them.  It’s 
> also
> about the time when Ricardo and I published
> <https://hal.inria.fr/hal-01161771/en> (Ricardo already had 
> experience
> with Guix in HPC and surely has an interesting story to share 
> :-)).
>
> We kept discussing these matters, together with sysadmins of the 
> local
> cluster, who were also looking for a solution they could offer 
> users in
> addition to modules.  Eventually Guix was installed on that 
> cluster and
> local HPC teams got into it at that point.
>
> That’s my story!  :-)
>
> It’s hopefully not the end of the story though.  There are other
> clusters listed at <https://hpc.guix.info/about/> that provide 
> Guix.
> We’re discussing with others in France, but these things take 
> time…
>
> > Yes I meant that. They run CentOS 8, so convincing them to run 
> > a build
> > daemon will not be as simple as `apt install guix`, but we'll 
> > see how
> > the conversation goes :)
>
> Tools like modules, Conda, Spack, and “containers” (Docker, 
> Singularity)
> are quite entrenched now in HPC.  All these tools are relatively 
> new so
> proposing yet another tool can be a hard sell.
>
> Also, cluster sysadmins tend to be conservative and don’t like 
> the idea
> of having a “daemon running as root”.
>
> But my impression is that there’s increasing awareness of the
> limitations of these tools.  It used to be that people would 
> disagree
> when we said in talks that Docker/Singularity images are not
> reproducible; now this is more widely understood.  Likewise for 
> the
> deployment issues around Jupyter and workflow languages.
>
> And there’s not a zillion tools out there to address these 
> issues.  :-)
>
> Good luck with your endeavors!
>
> Ludo’.