Re: Introducing Guix to HPC at my institution

[zimoun]

Hi Sábastien,

On Tue, 16 Mar 2021 at 10:54, Sébastien Lerique <sl@eauchat.org> wrote:

> On 15 Mar 2021 at 22:47, zimoun <zimon.toutoune@gmail.com> wrote:

> I am French, I'll be very happy to attend if I can! I live in
> Japan now though, so I guess I'll miss parts. Are you planning to
> record the workshop?

Cool!  If I remember correctly, there is some Japaneses’s users.  Do not
hesitate to roam on #guix (irc.freenode.net) or drop an email to
help-guix. :-)

We have not discussed the record of the workshop yet.  Keep you in
touch.

Well, I do not know if it is visible all around the world, here some
materials: 

<https://calcul.math.cnrs.fr/2021-01-anf-ust4hpc-2021.html#programme>

If I may, I would recommend the Konrad’s talk because it explains why
reproducibility matters for scientific reproducibility.  The next talk
by Ludo provides good connections with these principles and how to do in
practise.  I also recommend the 2 last talks by Francois Rué et Florent
Pruvost.  Other materials are worth to watch when speaking about
reproducibility, for sure, but they are not with a Guix angle.  Last, I
did not run the «TPs» so I cannot tell more.

Ludo’s did a nice talk at FOSDEM 2019 about distro:

<https://archive.fosdem.org/2019/schedule/event/gnu_guix_new_approach_to_software_distribution/>

well, he presents some arguments why and how Guix addresses the
challenge.  Moreover, here

<https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/talks>

you can find some raw materials used in various talks.  Especially one
at CERN in 2018 for instance:

<https://cds.cern.ch/record/2316926>



> Another thought: given Ludo's position, I am imagining that Inria
> Bordeaux runs Guix on some of their infrastructure, is that the
> case? If so, can you share any details about how that came to
> happen?

I cannot share how it runs at INRIA Bordeaux since I am far from my
beloved South West. :-) Well, some words how it runs at my place in a
small team of a Research Institute doing stuff in the biomedical field.

First, I am trained in apply maths and I studied wave propagation
(numerical methods, linear solver, etc.).  At the time, I am using
mainly Conda and some modulefiles prepared by the cluster’s team of the
different institutes I were.

Then, I traveled back to home and found a permanent position in this
Institute.  And I deal with Biologists with few skills on how to run
their computations.  I inherited a small cluster and couple of strong
desktop machines with an history.  Therefore, at the beginning, I was
doing the modulefiles by hand… then I said stop!  It does not scale
since I am alone and I felt as reinventing the wheel: repackaging
everything by hand.  Conda is still an option but Conda is doing wrong
at different levels.

I was following Guix, as the Emacs of distro. :-) And on Dec. 2018, Ludo
organized a day in Paris and I attended.  Then I installed Guix at work
on my main desktop and another remote strong desktop.  I teach Guix to
the new users and almost support only Guix.  But you know how people
have their habits. :-) One and half year later (2020 is off, since I was
recovering from a sport’s injury), I convinced people to switch to Guix
and I am in the process to reinstall the small cluster (for historical
reasons, LTS and never upgraded with too old kernel and now impossible
to upgrade “for free”, whatever!).

On strong desktops (256G of RAM, 64 cores) running a classic distro as
Ubuntu or Debian, installing Guix on the top is straightforward.  I do
as much as I can on the machines I am in charge.

Why Guix?  Beyond reproducibility, it is easy to have R packages via
“guix import”, for instance.  For the record, a Medical Doctor is
writing their own packages.  Well, people are less waiting after me and
they install the packages they need.  Guix works from laptop to cluster,
the surprises are really mitigated compared to the other tools I know.
Last, for users a bit afraid by the command line, I pack a Docker image
all included for them. Now a lot of journals ask code and data and I
would like to convince people in my Institute that
channels.scm+manifest.scm capture and Docker is just the container, easy
to move from one place to another; it has not happened yet.  That’s why
I investigated in things like:

<https://yhetil.org/guix/86bldahz42.fsf@gmail.com>


Last, Guix is a Scheme library.  Aside the parenthesis which is just not
an issue for a good editor, being Lisp allows to easily hack in.  Once
one is a bit familiar with a Lisp (Emacs Lisp, Racket, Common Lisp,
whatever, not necessary Guile) by running a tutorial, it is relatively
easy to read the code and understand which part does what.  For
instance, my first contribution after the usual packaging is fixing a
minor bug in the relevance scoring function.  I never understood Conda
enough to be able to fix any annoyance that I had.



> Yes I meant that. They run CentOS 8, so convincing them to run a
> build daemon will not be as simple as `apt install guix`, but
> we'll see how the conversation goes :)

Good luck!  Please report their feedback if they are not convinced.


> I'll write back here in a few days once I've thought about how to
> present things to the sysadmins. In the meantime, any feedback on
> past experience is very welcome!

I guess you already know the section «Clusters Deployment»:

<https://hpc.guix.info/about/>

Cheers,
simon