Re: MPI cluster and container

[Giovanni Biscuolo]

Hello zimoun,

zimoun <address@hidden> writes:

[...]

> Concerning the 1-, I mean the package manager part of Guix on a
> cluster with MPI configured.

*already having* a set of nodes you'd like to share the store on,
Ludovic wrote a nice article on how to do it:
https://guix-hpc.bordeaux.inria.fr/blog/2017/11/installing-guix-on-a-cluster/
it explains well what does it mean to "install Guix for system-wide use"
in a (possibly faked) cluster environment

Ricardo also wrote this
https://guix.mdc-berlin.de/documentation.html#sec-2
with less details (but the rest of the document is interesting as HPC
environment reference)

I've adapted Ludovic instructions to share the store with an LXC
container [1] via bind mounts (instead of NFS exports) of /gnu/store
(ro) and /var/guix (rw)... and it works [2]

for this to work I have to share both users (I do it via NIS) and their
homes with the container, in order to have the same user profiles on
both host and container

a similar approach should also work for QEMU/KVM VMs and other
virtualization software

please consider that nowadays we can avoid starting the guix-daemon in
TCP listening mode and use the default daemon setup: listen on the
shared socket on /var/guix/daemon-socket/socket on the "master node"/host and
use GUIX_DAEMON_SOCKET variable [3] on the "computing nodes"/guests, by
default (hardcoded?) pointing to the same socket

...aka you can skip the guix-daemon.service tweaks on host and
GUIX_DAEMON_SOCKET settings on guest


[...]

> Well, the idea is to create a fake MPI cluster with controlled
> containers. Does someone play around?

not for MPI but I'm starting to create a network of containers to test
networked services, but I'm still at early stages

IMHO - in our use case - a much more effective alternative is to use
a cluster of "guix system container"s [4] and/or "guix system vm"s

HTH!
Giovanni

[1] both host and LXC container are foreign distros, Debian 9 in my case

[2] sometimes I'm experiencing some GPU-related problems with GUIs but
it almost for sure depends on my "strange" environment: the LXC
container is also a LTSP server I'm using via a LTSP terminal... but
that's another story

[3] 
https://www.gnu.org/software/guix/manual/en/html_node/The-Store.html#index-GUIX_005fDAEMON_005fSOCKET

[4] please consider this non-blocking bug: 
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34211

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

signature.asc
Description: PGP signature