From MAILER-DAEMON Thu Apr 01 04:35:54 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRsny-0003wL-Fo for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 04:35:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40446) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRsnv-0003vS-HC for guix-science@gnu.org; Thu, 01 Apr 2021 04:35:51 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:11414) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRsni-00045x-Et for guix-science@gnu.org; Thu, 01 Apr 2021 04:35:50 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AO0EL7KGyzxOqKMajpLqECseALOonbusQ8zAX?= =?us-ascii?q?/mp2TgFYddHdqtC2kJ0gtSPcpT4NVBgb8+yoF7KHRRrnmKJdxak0EfOcUBL9uG?= =?us-ascii?q?2uRbsC0aLHzyf7EyPzss5xvJ0OT4FEBNf9DUd3gK/BiWHSL/8b3NKF/Kq07N2x?= =?us-ascii?q?815RTBpnY6wlzwB1BhfzKDwSeCB6BIE0HJfZ28xfpzDIQwVyUu2HABA+M9Trlp?= =?us-ascii?q?nunJLibQVuPW9E1CC+yRWy6LD7FBCE3hARFxN3qI1SklT4rw=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,296,1610406000"; d="scan'208";a="501061986" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Apr 2021 10:35:32 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: =?utf-8?Q?S=C3=A9bastien?= Lerique Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> <87blb1wgna.fsf@eauchat.org> <87o8f1jee2.fsf@inria.fr> <87ft0bvqv5.fsf@eauchat.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 10:35:32 +0200 In-Reply-To: <87ft0bvqv5.fsf@eauchat.org> (=?utf-8?Q?=22S=C3=A9bastien?= Lerique"'s message of "Wed, 31 Mar 2021 14:23:10 +0900") Message-ID: <87mtui1jxn.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 08:35:51 -0000 Howdy! S=C3=A9bastien Lerique skribis: > Case 1: with substitutes, on a local (non-NFS) folder > ----------------------------------------------------- > > $ guix archive --authorize < bin/../share/guix/ci.guix.gnu.org.pub > $ guix-daemon --disable-chroot & > $ guix install hello > [... dowloading ...] > unexpected substituter message 'defining `GUIX_LOCPATH', along > these lines: > > guix install glibc-utf8-locales > export GUIX_LOCPATH=3D"$HOME/.guix-profile/lib/locale" > > See the "Application Setup" section in the manual, for more info. > ' > > I think this is https://issues.guix.gnu.org/45166. > > Using instead `guix pack -R -S /bin=3Dbin -S /etc=3Detc -S /lib=3Dlib guix > bash glibc-utf8-locales`, and: > > $ export GUIX_LOCPATH=3D$(pwd)/lib/locale > $ ls $GUIX_LOCPATH/ > 2.31 > > seems to start working, but then `guix install hello` fails with the > same error, and: > > $ ls $GUIX_LOCPATH/ > ls: cannot access '/tmp/sebastien-lerique/guix/lib/locale/': No > such file or directory > > Is it possible that glibc-utf8-locales gets garbage-collected? Can you run guix-daemon like so? LC_ALL=3Den_US.utf8 guix-daemon --disable-chroot & The =E2=80=9Cunexpected substitute message=E2=80=9D thing is a bug: it turn= s out that stderr of =E2=80=98guix substitute=E2=80=99 is consumed directly by the dae= mon at this point, which it shouldn=E2=80=99t (similar to the issue fixed by ee3226e9d54891c7e696912245e4904435be191c). > Case 2: from source (no substitutes), on a local (non-NFS) folder > ----------------------------------------------------------------- > > No need for the locale fix here (another bug appears before that): > > $ guix-daemon --disable-chroot & > # --no-offload works around the missing nscd problem; up to now > this > # doesn't seem necessary when using substitutes > $ guix install hello --no-offload > [... builds for a while ...] > build of > /gnu/store/pkn1w1q3xkn273kpmggc4dnq6n6hr9jy-bzip2-mesboot-1.0.8.drv=20 > failed > > The build log for bzip2-mesboot ends with: > > tcc -ar cq libbz2.a blocksort.o huffman.o crctable.o randtable.o > compress.o decompress.o bzlib.o > ranlib libbz2.a > /gnu/store/prkqai3zwh3shlqpll6xyncmmqpj49dd-gash-boot-0.2.0/bin/sh: > ranlib: Command not found. > make: *** [libbz2.a] Error 127 > command "make" "CC=3Dtcc -I ." "AR=3Dtcc -ar" "bzip2" > "PREFIX=3D/gnu/store/s94hyrv1vgllxir5niiyzfc9g80l5kcd-bzip2-mesboot-1.0.= 8"=20 > failed with status 2 > > Is this new and should be reported as a bug? Should binutils-mesboot0 > be part of bzip2-mesboot's inputs? (I haven't learned about the > boostrapping mechanism yet.) The problem is that =E2=80=98--disable-chroot=E2=80=99 is a bit of the wild= west: build processes can access the whole file system and what you do as a user can interfere with them. It could be that the bzip2 build failure above is just that: the build process picks something from /usr/lib or /usr/bin, and that breaks everything. I think =E2=80=98--disable-chroot=E2=80=99 is OK if you=E2=80=99re going to= use substitutes for almost everything. Otherwise, it=E2=80=99s not good. Your use case calls = for built-in support; that way, the daemon could take still advantage of user namespaces to set up a chroot and all. > Case 3: with substitutes, on an NFS share > ----------------------------------------- > > Again no need for the locale fix (another bug appears before that): > > $ guix archive --authorize < bin/../share/guix/ci.guix.gnu.org.pub > $ guix-daemon --disable-chroot & > $ guix install hello > [... downloading ...] > guix install: error: cannot unlink > `/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib':=20 > Directory not empty > > so checking: > > $ ls -lA /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib > total 3840 > -r-xr-xr-x 1 sebastien-lerique froeseuni 64832 Jan 1 1970 > .nfs000000000172caa5000054bd > [... more .nfs files ...] Ah yes, that=E2=80=99s a known issue with NFS. I=E2=80=99m not aware of an= y workaround. Thanks, Ludo=E2=80=99. From MAILER-DAEMON Thu Apr 01 06:17:34 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRuOM-0000pW-01 for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 06:17:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRuOH-0000os-TK for guix-science@gnu.org; Thu, 01 Apr 2021 06:17:30 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:22210) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRuOD-0005aA-Of for guix-science@gnu.org; Thu, 01 Apr 2021 06:17:29 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3Ajo6zE6xfDhklKqP53An6KrPw571zdoIgy1kn?= =?us-ascii?q?xilNYDZSddGVkN3roe8S0gX6hC1UdHYrn92BP6foewK/ybde544NMbC+GDT3oW?= =?us-ascii?q?fAFu9fxKbr3jGIIU3D38FH06MISdkcNPTVLXxXyfn3+xO5FdFI+ra62YSln/3X?= =?us-ascii?q?wXsobQwCUcBdxjx0AAqaDUF6LTMubfFSKLOn+sFFqzC8EE56Uu2HABA+MtT+mw?= =?us-ascii?q?=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,296,1610406000"; d="scan'208";a="501092201" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Apr 2021 12:17:23 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: guix-science@gnu.org Subject: GC strategy on clusters X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 12:17:22 +0200 Message-ID: <87v9969uml.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 10:17:30 -0000 Hi there! Recently the Guix head node of our cluster at Inria was getting short on disk space, despite running =E2=80=98guix gc -F20G=E2=80=99 (or similar) tw= ice a day. Turns out that some users had accumulated many profile generations and that was getting in the way. So we kindly asked them to run: guix package --delete-generations=3D4m or similar, which was enough to free more space. We=E2=80=99re now considering setting up automatic user notification by ema= il, as is commonly done for disk quotas, asking them to remove old generations. That way, users remain in control and choose what GC roots or generations they want to remove. How do people on this list deal with that? Longer term, I think Guix should automatically delete old generations and instead store the channels + manifest to reproduce them, when possible. Ludo=E2=80=99. From MAILER-DAEMON Thu Apr 01 08:36:58 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRwZF-00060n-Vh for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 08:36:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52528) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRwZE-0005z2-Eh for guix-science@gnu.org; Thu, 01 Apr 2021 08:36:56 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:4309) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRwZC-0008Fd-2n for guix-science@gnu.org; Thu, 01 Apr 2021 08:36:55 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AGHwtO6qQ2S/uKFsd9/cx48EaV5s6LNV00zAX?= =?us-ascii?q?/kB9WHVpW+SCncGvg/gXkTfo4QxhGU0Is9aGJaWGXDfg5Yd4iLNhRYuKcQH6tA?= =?us-ascii?q?KTXeNfxKf42Dv6AWnC8IdmpNJdWodkDtmYNykese/e4BOkV/46ytiG76zAv5aR?= =?us-ascii?q?815JTRt2L4Bt6h4RMHfRLmRSRBNaQbo1faDsrvZvgjq7ZDAwcMO2HRA+Lqf+jv?= =?us-ascii?q?jKjo+jWBgdGnccmUWzpBap8qO/KQSSxB0EUzhCqI1Su1Tttgzi++GetOumwQXX?= =?us-ascii?q?zG/Z4/1t6Zbc4/9iIODJtcQPMDXrjW+TFf1ccpmPpio8ru3qyHtCqqirnz4YJM?= =?us-ascii?q?h4537ceWu4yCGdr3iW7B8U53XvyUCVjBLYyKSTLl5UN+N7iZtdYl/l7SMbzbVB?= =?us-ascii?q?+ZhG2G+UrPNsfGX9tR7g7NvFXQwCrDvFnVMel4co/htieLpbQKNNrIAZ4UNeGI?= =?us-ascii?q?oBGiW/1og9F+loAqjnicp+QBe+aXbB11Mfs+CEbzAWMlOvXlUC0/b17xFm2FRU?= =?us-ascii?q?i2YR2cB3pBc93aN4b7UBxujANM1T+o1mf4szQeZcPY46MKiKI12IZwnNPmKUKU?= =?us-ascii?q?nmE69CG0ulke+N3JwFoNunYIAFy5E/8a6xIm9whCoJYkrrTeeO0JdI/guIf2mn?= =?us-ascii?q?RzTqzahllulEk4y5fqPiNWmnRE8jlqKb0osiK/yeYe+6PNZtHvf/MXGGI/c34y?= =?us-ascii?q?TOH6dvDT0wa+sw0+xLIW6mk4b0FqmvjcDjVcbvAtPWYEoZc1K6PEcmGAXUTf8w?= =?us-ascii?q?k3yDazv+hlzYQHntelPylKgAWZTnww=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,296,1610406000"; d="scan'208";a="377511581" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Apr 2021 14:36:50 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Efraim Flashner Cc: guix-science@gnu.org Subject: Re: GC strategy on clusters References: <87v9969uml.fsf@inria.fr> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 14:36:50 +0200 In-Reply-To: (Efraim Flashner's message of "Thu, 1 Apr 2021 13:45:37 +0300") Message-ID: <87wntm89lp.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.104; envelope-from=ludovic.courtes@inria.fr; helo=mail3-relais-sop.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 12:36:56 -0000 Hello! Efraim Flashner skribis: > On Thu, Apr 01, 2021 at 12:17:22PM +0200, Ludovic Court=C3=A8s wrote: [...] >> guix package --delete-generations=3D4m >>=20 >> or similar, which was enough to free more space. > > I feel like 4-6 months should be plenty for anything active. Even if it > were run automatically for them it wouldn't remove the last generation. It depends. A practical use case I have in mind: you run experiments, you submit a paper including its results, you get initial reviews months later, and even later it=E2=80=99s published and you get to present it. At= that point, you want to answer questions and to reproduce it. 4=E2=80=936 month= s is not a lot in that context. (Though of course, ideally you=E2=80=99d save channels.scm + manifest.scm a= nd share it with reviewers and readers in the first place=E2=80=A6) Besides, I think the whole point of Guix is that users on the cluster can remain in control, unlike what happens with =E2=80=9Cenvironment module= s=E2=80=9D. >> Longer term, I think Guix should automatically delete old generations >> and instead store the channels + manifest to reproduce them, when >> possible. >>=20 > > This seems to help a bit less when we run into issues about dates being > wrong on SSL tests, or when sources go missing. Good points. Hopefully =E2=80=9Csources go missing=E2=80=9D can soon be co= nsidered addressed. Really, failing TLS tests is the most worrisome issue to me because we don=E2=80=99t have any idea on how to address it systematically. > How much storage and people are you working with? Our initial multiuser > system has 188GB for /gnu and I think 30-40 people and some people have > profiles going back almost 3 years. Not many people have multiple > profiles and the experiments we tried with shared profiles in > /usr/local/guix-profiles don't see a lot of use or get updated > frequently. I=E2=80=99m not sure how much storage the Guix head node has (I=E2=80=99m n= ot an admin), but the number of users and duration is in the same ballpark. > I guess I'm not really sure if its a technology problem or a people > problem. Figuring out if someone is the only one pulling in a copy of > glibc-2.25 is doable but how many copies of diffoscope is too many? > > On a practical note, 'guix package --list-profiles' as root currently > lists everyone's profiles so it can be easier to see who has older > profiles hanging around. Actually, as non-root, I walked /var/guix/profiles/per-user on the cluster to see the number of generations per user, which allowed us to target those with a lot of generations. :-) It would be nice to provide a documented approach sysadmins could follow! Ludo=E2=80=99. From MAILER-DAEMON Thu Apr 01 09:29:59 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRxOX-00030X-9U for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 09:29:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41724) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRxOV-0002x5-CP for guix-science@gnu.org; Thu, 01 Apr 2021 09:29:55 -0400 Received: from mail-qk1-x72f.google.com ([2607:f8b0:4864:20::72f]:41612) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lRxOR-0004u5-Ku for guix-science@gnu.org; Thu, 01 Apr 2021 09:29:54 -0400 Received: by mail-qk1-x72f.google.com with SMTP id v70so2115623qkb.8 for ; Thu, 01 Apr 2021 06:29:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=gtxRpWtxnpbVwxKypHCeWWRd/QTjBcva/DmIgsM2qac=; b=fmGLs3cHL0yfkRch1BhTrz/TE4oABHHCJ/I+68JRsMLJAG7fVYLQjndsKejxgmCq7b +VosdSfBj86yQdMo9XRYN5QFtVqIL8NP79YVH9sAWZJ2Fke/k2ZAM+Unvgbd79ZSMfGW /0rdfmCzEMe4zBTAU6cle2z0tJJ87TTFglIx7hNJQBrcIoRb9gnV5E9yskRbCAACB8Yw q0rN6NkNNxXdM3SpqGvoJTlQdUqJ6ZgrHrcLKqfeiX7NhIlRlJrjpxzH+9gPLQToKNny g55IDjctAWwcLnO6fPNZz4JbyMn5qI7WrwU5/6agNy3cn2DabnGICNWokxakHkdL1Pu9 EODA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=gtxRpWtxnpbVwxKypHCeWWRd/QTjBcva/DmIgsM2qac=; b=ShTCGKEc5+Cmzgl24qZqnYfPnlj4YAY3+5qmtiuU8kV6qHt6JOe+EEuxjXDR3/pCmk UcLyAlUX6NVEZhtuL+Y4tQSCZ1e2LBhxbKfjZzHOO7UzdSkSO6q5O9HPExZJOnzUwglc XzLETO526cTE/zMdvwadhHjxY8ZojK2lsM0BFTMy+eC09iZKFiFkMy1fG8YE/CQ8WMkV SgCQDwifqrNfpiSfZJ9c12Jj9BbpA+oO08ZwLRbVGAfnWP55k1XrCw1pdklrhLl1TDZq ELY6b2CyrKL5shad2h1YnC0wvWyNUAKh5KxMgddUg+nYRkx+lJrdvz4mrymG0ZfrcCBs KYwg== X-Gm-Message-State: AOAM530y7wx0oOLJpd+TRKRf4ywXE7Xzn3miRExZnYCK6+nxXtCmH/qN ozBFq89cRICTN+sN4Jlcq9ejRZbsh+zHiC3bbgY= X-Google-Smtp-Source: ABdhPJy5NJG5/nYa3coAJiCiWWcr2bvrct2+K8PxKM1kWiNLPFl2SzIk4eWBzNbDXypd8pTE+wa/d1VVbY4RuOW28ug= X-Received: by 2002:a37:b807:: with SMTP id i7mr8179441qkf.126.1617283789536; Thu, 01 Apr 2021 06:29:49 -0700 (PDT) MIME-Version: 1.0 References: <87v9969uml.fsf@inria.fr> <87wntm89lp.fsf@inria.fr> In-Reply-To: <87wntm89lp.fsf@inria.fr> From: zimoun Date: Thu, 1 Apr 2021 15:29:38 +0200 Message-ID: Subject: Re: GC strategy on clusters To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Cc: Efraim Flashner , guix-science@gnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::72f; envelope-from=zimon.toutoune@gmail.com; helo=mail-qk1-x72f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 13:29:55 -0000 Hi, On Thu, 1 Apr 2021 at 14:36, Ludovic Court=C3=A8s wrote: > Efraim Flashner skribis: > > > On Thu, Apr 01, 2021 at 12:17:22PM +0200, Ludovic Court=C3=A8s wrote: > > [...] > > >> guix package --delete-generations=3D4m > >> > >> or similar, which was enough to free more space. For now, I am waiting to reach the full capacity at hand to ask how to deal with it. ;-) > > I feel like 4-6 months should be plenty for anything active. Even if it > > were run automatically for them it wouldn't remove the last generation. > > It depends. A practical use case I have in mind: you run experiments, > you submit a paper including its results, you get initial reviews months > later, and even later it=E2=80=99s published and you get to present it. = At that > point, you want to answer questions and to reproduce it. 4=E2=80=936 mon= ths is > not a lot in that context. To add another data point. Even it is hard to have a good overview, I would say the average is 2-4 years for a typical project. It is hard because currently all is not done with the same tools for the same task. For instance, some data is cleaned with some tools, then an partial analysis is done, months later another data is added so another partial analysis with probably different tools, then months later a full toolsuite as Bioconductor (or whatever) is updated and some analysis are re-done. The final publication is a mix of all over the 2-4 years project with details at various level. Other said, it depends on the level we are looking. > (Though of course, ideally you=E2=80=99d save channels.scm + manifest.scm= and > share it with reviewers and readers in the first place=E2=80=A6) It is what I am tirelessly explaining in my lab. ;-) > >> Longer term, I think Guix should automatically delete old generations > >> and instead store the channels + manifest to reproduce them, when > >> possible. > >> > > > > This seems to help a bit less when we run into issues about dates being > > wrong on SSL tests, or when sources go missing. > > Good points. Hopefully =E2=80=9Csources go missing=E2=80=9D can soon be = considered > addressed. Really, failing TLS tests is the most worrisome issue to me > because we don=E2=80=99t have any idea on how to address it systematicall= y. By "soon", you mean the bricks are there and it is missing to glue them together. From my opinion, some details need to be addressed to have a full end-to-end sources fallback, since evil is hidden inside the details. ;-) About the TLS, you proposed to setup a machine ahead of clock. Maybe it is worth to try. > > I guess I'm not really sure if its a technology problem or a people > > problem. Figuring out if someone is the only one pulling in a copy of > > glibc-2.25 is doable but how many copies of diffoscope is too many? > > > > On a practical note, 'guix package --list-profiles' as root currently > > lists everyone's profiles so it can be easier to see who has older > > profiles hanging around. > > Actually, as non-root, I walked /var/guix/profiles/per-user on the > cluster to see the number of generations per user, which allowed us to > target those with a lot of generations. :-) Well, when we discussed the '--list-profiles', it was initially for my personal purposes. Then, I have tried to use it to monitor the few users that I have. Well, in Biology they have the concept of "-80 fridge". It is a big and very cold fridge where you keep samples, potentially for a long time. Everybody put in until it is full and once it is full, there is endless discussion on what to throw... until the fridge is broken because shutdown or unexpected failures and then it is obvious to everybody what needs to be taken or thrown. I am using such analogy to explain the hygiene to have on shared machines Hum, once written I do not know if it relevant. ;-) >From my point of view, having a channel+manifest "backup" for old profiles seems something to try. It cost nothing with ''--export-manifest" and "--export-channels". Cheers, simon From MAILER-DAEMON Thu Apr 01 09:43:57 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRxc4-00040g-Vr for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 09:43:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46384) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRxc3-0003yI-39 for guix-science@gnu.org; Thu, 01 Apr 2021 09:43:55 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:49615) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRxc0-0004qe-KM for guix-science@gnu.org; Thu, 01 Apr 2021 09:43:54 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3ATbmSd6EhStF5Lx2epLqFY5HXdLJzesId70hD?= =?us-ascii?q?6mlYVQFSNvaFnMSvgekW2HbP+UkscVsnns2NP7TFfGjV8oR77ZJUEbCpWgTnv2?= =?us-ascii?q?XAFvAG0aLJxTr8FyristNMzKsISdkdNPTcBUV35PyQ3CCWCNAlqePszImJgqPk?= =?us-ascii?q?w25pXUVWbchbgTtRLwaADwlXSWB9dOMEPb+d/NcCmz27ZX8MZN+6DXVtZZm6m/?= =?us-ascii?q?TumIj9aRALQz4LgTPgsRqN6Lr3VyeVxQ0fVTQn+8ZAzUH/nwb05rquvpiAo3fh?= =?us-ascii?q?/lTe4JhMlNzqxsErPq2xo/MYMzD9zjuvDb4NZ5S5oDs3rOuzgWxLrPDwpX4bU/?= =?us-ascii?q?hb2jf4Ynyy5SDgwRDn1ywjr0X/0EaCjWH4yPaJOg4SOo5uhZ99ehCcz0Y6st1G?= =?us-ascii?q?3LpW03iF87p7ZCmw5xjV1pznVwxOnkHxhXY4n+Ipg2RCXZECLJ9wgOUkjTloOa?= =?us-ascii?q?ZFMiLmzYgtVM5pEcHS5PxfbFOdaEvUtnNox7WXLwkONybDbEgen8SflwFMmnN0?= =?us-ascii?q?xVZw/r16ol4wsLw8VrxN7KD8KaJqmLtSCu8aBJgNY9spcI+YCnHpSRmJCXmVIl?= =?us-ascii?q?jsCeU/M2/NwqSHhokd1aWFeIEsxJB3o5jZVUMdl2IjEniedfGz4A=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,296,1610406000"; d="scan'208";a="501142978" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Apr 2021 15:43:46 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zimoun Cc: Efraim Flashner , guix-science@gnu.org Subject: Re: GC strategy on clusters References: <87v9969uml.fsf@inria.fr> <87wntm89lp.fsf@inria.fr> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 15:43:45 +0200 In-Reply-To: (zimoun's message of "Thu, 1 Apr 2021 15:29:38 +0200") Message-ID: <87zgyi6rxq.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 13:43:55 -0000 Hi, zimoun skribis: >> It depends. A practical use case I have in mind: you run experiments, >> you submit a paper including its results, you get initial reviews months >> later, and even later it=E2=80=99s published and you get to present it. = At that >> point, you want to answer questions and to reproduce it. 4=E2=80=936 mo= nths is >> not a lot in that context. > > To add another data point. Even it is hard to have a good overview, I > would say the average is 2-4 years for a typical project. It is hard > because currently all is not done with the same tools for the same > task. For instance, some data is cleaned with some tools, then an > partial analysis is done, months later another data is added so > another partial analysis with probably different tools, then months > later a full toolsuite as Bioconductor (or whatever) is updated and > some analysis are re-done. The final publication is a mix of all over > the 2-4 years project with details at various level. Right. >> Good points. Hopefully =E2=80=9Csources go missing=E2=80=9D can soon be= considered >> addressed. Really, failing TLS tests is the most worrisome issue to me >> because we don=E2=80=99t have any idea on how to address it systematical= ly. > > By "soon", you mean the bricks are there and it is missing to glue > them together. From my opinion, some details need to be addressed to > have a full end-to-end sources fallback, since evil is hidden inside > the details. ;-) True! I=E2=80=99m really hopeful about the Disarchive/SWH combination, tog= ether with . Of course we=E2=80=99ll have to monitor that, and we can expect bumps on the road :-), but at least we have a plan. > About the TLS, you proposed to setup a machine ahead of clock. Maybe > it is worth to try. Oh sure, that trick definitely works. But it=E2=80=99s a terrible hack, an= d it means that, by default, people will just fail to build the package. > Well, when we discussed the '--list-profiles', it was initially for my > personal purposes. Then, I have tried to use it to monitor the few > users that I have. Well, in Biology they have the concept of "-80 > fridge". It is a big and very cold fridge where you keep samples, > potentially for a long time. Everybody put in until it is full and > once it is full, there is endless discussion on what to throw... until > the fridge is broken because shutdown or unexpected failures and then > it is obvious to everybody what needs to be taken or thrown. I am > using such analogy to explain the hygiene to have on shared machines > Hum, once written I do not know if it relevant. ;-) It surely is. :-) I mean, I didn=E2=80=99t even think about it until GC couldn=E2=80=99t make any progress. > From my point of view, having a channel+manifest "backup" for old > profiles seems something to try. It cost nothing with > ''--export-manifest" and "--export-channels". Yes, though it=E2=80=99s an approximation; so it should only be used when we know that it=E2=80=99s 100% faithful, as is the case for =E2=80=98guix pull= =E2=80=99 profiles. Ludo=E2=80=99. From MAILER-DAEMON Thu Apr 01 10:35:37 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRyQ5-0005qv-6O for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 10:35:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60356) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRyQ3-0005oF-5h for guix-science@gnu.org; Thu, 01 Apr 2021 10:35:35 -0400 Received: from elegua.eauchat.org ([91.224.149.118]:46104 helo=eauchat.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lRyQ0-0003LV-UP for guix-science@gnu.org; Thu, 01 Apr 2021 10:35:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eauchat.org; s=mail; t=1617287728; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=epsfmRfqGAoGTy4ZbWoRY4RV4f+kl7PYaOkR5V4qmhs=; b=IZdETDMKOKWGQe6ohQE7dK+UYC3myjWOgf2czgRLwcCGxfpQW9KORwqtnze0bCgI9FwG6I nAXvX2zpUFCVmBONGgxbvhR0mYQeMW6hr01HTjncyExLPUTsrGmplwBXu0VYVnZ/uvVmcz OwfhCvkFK0uhMGoEDNi2S1Ry0DctY48= References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> <87blb1wgna.fsf@eauchat.org> <87o8f1jee2.fsf@inria.fr> <87ft0bvqv5.fsf@eauchat.org> <87mtui1jxn.fsf@gnu.org> From: =?utf-8?Q?S=C3=A9bastien?= Lerique To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution In-reply-to: <87mtui1jxn.fsf@gnu.org> Message-ID: <87k0pmysyk.fsf@eauchat.org> Date: Thu, 01 Apr 2021 23:34:12 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=91.224.149.118; envelope-from=sl@eauchat.org; helo=eauchat.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 14:35:35 -0000 Hello! On 01 Apr 2021 at 17:35, Ludovic Court=C3=A8s=20 wrote: >> Case 1: with substitutes, on a local (non-NFS) folder >> ----------------------------------------------------- >> [snip] > > Can you run guix-daemon like so? > > LC_ALL=3Den_US.utf8 guix-daemon --disable-chroot & Then: $ guix install hello --no-offload [... dowloading ...] 13.3 MB will be downloaded substitution of=20 /gnu/store/395pvii4bcjqxvdv7h0drq10lxi01sv1-glibc-utf8-locales-2.31=20 failed guix install: error: some substitutes for the outputs of=20 derivation=20 `/gnu/store/b2jkmg71m0dpf3i6hvskb32ra48lls28-glibc-utf8-locales-2.31.drv'= =20 failed (usually happens due to networking issues); try=20 `--fallback' to build derivation from source Restarting from scratch and using `--fallback` then leads to the=20 bzip2-mesboot failure of Case 2. If, instead of resarting from scratch, I just run again `guix=20 install hello --no-offload`, I then get: guix install: error: got unexpected path=20 `/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash= :=20 warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)'=20 from substituter And indeed once again lib/locale is a dead symlink, as the=20 glibc-utf8-locales packaged in my tar.gz has been removed from the=20 store. Why is that? I had a look through the same commands with=20 --debug=3D4 or 5, but to no insight (aside from seeing locale=20 derivations being deleted at some stage). I posted the end of the=20 log produced by `guix install hello --no-offload --debug=3D4`,=20 starting with the failed substitution, here:=20 https://paste.debian.net/1191991/ . > The =E2=80=9Cunexpected substitute message=E2=80=9D thing is a bug: it tu= rns out=20 > that > stderr of =E2=80=98guix substitute=E2=80=99 is consumed directly by the d= aemon=20 > at this > point, which it shouldn=E2=80=99t (similar to the issue fixed by > ee3226e9d54891c7e696912245e4904435be191c). Ok, I'll file a bug for that then :) >> Case 2: from source (no substitutes), on a local (non-NFS)=20 >> folder >> ----------------------------------------------------------------- >> [snip] > > The problem is that =E2=80=98--disable-chroot=E2=80=99 is a bit of the wi= ld=20 > west: build > processes can access the whole file system and what you do as a=20 > user can > interfere with them. > > It could be that the bzip2 build failure above is just that: the=20 > build > process picks something from /usr/lib or /usr/bin, and that=20 > breaks > everything. > > I think =E2=80=98--disable-chroot=E2=80=99 is OK if you=E2=80=99re going = to use=20 > substitutes for > almost everything. Otherwise, it=E2=80=99s not good. Your use case=20 > calls for > built-in support; that way, the daemon could take still=20 > advantage of > user namespaces to set up a chroot and all. I see. By the way, starting guix-daemon without `--disable-chroot`=20 (and without substitutes) leads to: guix build: error: cannot change ownership of=20 =E2=80=98/gnu/store/0dn61y4n8ig333b23hmc80hvlcy8gdli-guile-bootstrap-2.0.= drv.chroot=E2=80=99:=20 Invalid argument so --disable-chroot is indeed still necessary! I am interested in=20 working on this built-in support once I get the substitutes case=20 working (although I will probably come ask for guidance for that)=20 :) Thanks! S=C3=A9bastien From MAILER-DAEMON Thu Apr 01 11:25:39 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRzCV-0005MF-0J for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 11:25:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRurz-00071O-QK for guix-science@gnu.org; Thu, 01 Apr 2021 06:48:11 -0400 Received: from flashner.co.il ([178.62.234.194]:46970) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRuq3-0004Fw-Ay for guix-science@gnu.org; Thu, 01 Apr 2021 06:48:11 -0400 Received: from localhost (unknown [31.210.177.71]) by flashner.co.il (Postfix) with ESMTPSA id E32EA402D4; Thu, 1 Apr 2021 10:46:09 +0000 (UTC) Date: Thu, 1 Apr 2021 13:45:37 +0300 From: Efraim Flashner To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org Subject: Re: GC strategy on clusters Message-ID: References: <87v9969uml.fsf@inria.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="8wbxJonQkwZVtmsy" Content-Disposition: inline In-Reply-To: <87v9969uml.fsf@inria.fr> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Received-SPF: pass client-ip=178.62.234.194; envelope-from=efraim@flashner.co.il; helo=flashner.co.il X-Mailman-Approved-At: Thu, 01 Apr 2021 11:25:34 -0400 X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 10:48:12 -0000 --8wbxJonQkwZVtmsy Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 01, 2021 at 12:17:22PM +0200, Ludovic Court=C3=A8s wrote: > Hi there! >=20 > Recently the Guix head node of our cluster at Inria was getting short on > disk space, despite running =E2=80=98guix gc -F20G=E2=80=99 (or similar) = twice a day. >=20 > Turns out that some users had accumulated many profile generations and > that was getting in the way. So we kindly asked them to run: >=20 > guix package --delete-generations=3D4m >=20 > or similar, which was enough to free more space. I feel like 4-6 months should be plenty for anything active. Even if it were run automatically for them it wouldn't remove the last generation. > We=E2=80=99re now considering setting up automatic user notification by e= mail, > as is commonly done for disk quotas, asking them to remove old > generations. That way, users remain in control and choose what GC roots > or generations they want to remove. >=20 > How do people on this list deal with that? I like the idea of asking people to remove old generations. It's not something that we've come up against yet. It doesn't feel that different than reminding them that their $HOME is for code and smaller things and the storage space is for their large data collections. > Longer term, I think Guix should automatically delete old generations > and instead store the channels + manifest to reproduce them, when > possible. >=20 This seems to help a bit less when we run into issues about dates being wrong on SSL tests, or when sources go missing. How much storage and people are you working with? Our initial multiuser system has 188GB for /gnu and I think 30-40 people and some people have profiles going back almost 3 years. Not many people have multiple profiles and the experiments we tried with shared profiles in /usr/local/guix-profiles don't see a lot of use or get updated frequently. I guess I'm not really sure if its a technology problem or a people problem. Figuring out if someone is the only one pulling in a copy of glibc-2.25 is doable but how many copies of diffoscope is too many? On a practical note, 'guix package --list-profiles' as root currently lists everyone's profiles so it can be easier to see who has older profiles hanging around. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --8wbxJonQkwZVtmsy Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmBlpE8ACgkQQarn3Mo9 g1HHyRAAuYjmwQ1cpCosAMuksIzT5+egLOO2b/WKQapORcZIQcAdNyJAn4bMlNA1 mE73RiuCm3dGK6rGgZbiYHihPGOard2kS8xYS+kYr9r9oeAE7QYeoUj5OgtCgdbb /Oc19rb8EkbUXy1QGmzzglIkTBrgTDPqJD1k51aUluwATREFIgQW/E0GUTUXL/Hm kkiYwbb9o1SuUPQzokq2IIn/QEnnr25BQiDD/m52jatDktpBE24MwiOWzc9i/JRy 84dfo3xoQE0a7n+2W7pfEkXkwMf3PJtpLICeqVrAS+yCGE35KF6wYUVoCzQvfVl0 3tqvgnV+oHNFcelAgiVIdF1Dep6q0kCrMoz5AOfEDX1CX1ReUd8IhbjKataGeqqq rFQ4ebuecESOy5fIkDiFRAXBRVCVXTiWgdDn4RYR4KfcjjJJ37sDyseJxhoerGXR aAcmeAZ/lMpxTgi9V5fjilU37/wn2M+BrIxCsGJG7kWXZb+EQY+KTNweT/+Mj5Nn 58bbcvjH/VRjL5VIhC7vY9+xFye0c2LOIlzwrifuo0AV0nLr0M9CK4vRsRgi8T/7 RjuH+SXlvn1lQWnyPv5HfYeHKyN16VIUVPSHVg4/qefby0zZPlzD77L7l74A7KBL 4urQzvnZ64NW+n3GHkx5y8MCdYTtzoFDeDUbnMJEbhZfdNatmGs= =l2Fp -----END PGP SIGNATURE----- --8wbxJonQkwZVtmsy-- From MAILER-DAEMON Thu Apr 01 11:25:40 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lRzCW-0005OC-6e for mharc-guix-science@gnu.org; Thu, 01 Apr 2021 11:25:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38008) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRyix-0008Cx-Cg for guix-science@gnu.org; Thu, 01 Apr 2021 10:55:07 -0400 Received: from smtp.hosts.co.uk ([85.233.160.19]:10173) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRyiv-0005dF-Iy for guix-science@gnu.org; Thu, 01 Apr 2021 10:55:07 -0400 Received: from maikeh336.claranet.co.uk ([79.123.23.187] helo=pancake.local) by smtp.hosts.co.uk with esmtpsa (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim) (envelope-from ) id 1lRyip-0002TV-4l for guix-science@gnu.org; Thu, 01 Apr 2021 15:54:59 +0100 Message-ID: <1cb914b16ee0dad6bf5102bb4b66fda3b020dbec.camel@tourbillion-technology.com> Subject: reproducibility of numerical experiments From: Paul Garlick To: guix-science@gnu.org Date: Thu, 01 Apr 2021 15:54:58 +0100 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Received-SPF: none client-ip=85.233.160.19; envelope-from=pgarlick@tourbillion-technology.com; helo=smtp.hosts.co.uk X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Thu, 01 Apr 2021 11:25:33 -0400 X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 14:55:07 -0000 Hi Guix-Science, I gave a talk to the FEniCS finite element community last week at the FEniCS 2021 conference. The title was 'On the reproducibility of numerical experiments with FEniCS'. I introduced GNU Guix, described the reproducible build process and gave an example of a user profile defined by a manifest and channel specification. The conference was online this year. The slides can be found at: https://mscroggs.github.io/fenics2021/talks/garlick.html Best regards, Paul. From MAILER-DAEMON Fri Apr 02 11:11:11 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lSLS3-0001vd-B2 for mharc-guix-science@gnu.org; Fri, 02 Apr 2021 11:11:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46652) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lSLS1-0001vW-3p for guix-science@gnu.org; Fri, 02 Apr 2021 11:11:09 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:63500) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lSLRy-0002Dq-UK for guix-science@gnu.org; Fri, 02 Apr 2021 11:11:08 -0400 X-IronPort-AV: E=Sophos;i="5.81,300,1610406000"; d="scan'208";a="377637706" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Apr 2021 17:11:01 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Paul Garlick Cc: guix-science@gnu.org Subject: Re: reproducibility of numerical experiments References: <1cb914b16ee0dad6bf5102bb4b66fda3b020dbec.camel@tourbillion-technology.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 13 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 02 Apr 2021 17:11:01 +0200 In-Reply-To: <1cb914b16ee0dad6bf5102bb4b66fda3b020dbec.camel@tourbillion-technology.com> (Paul Garlick's message of "Thu, 01 Apr 2021 15:54:58 +0100") Message-ID: <87mtug203e.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.104; envelope-from=ludovic.courtes@inria.fr; helo=mail3-relais-sop.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2021 15:11:09 -0000 Hi Paul, Paul Garlick skribis: > I introduced GNU Guix, described the reproducible build process and > gave an example of a user profile defined by a manifest and channel > specification. > > The conference was online this year. > > The slides can be found at: > > https://mscroggs.github.io/fenics2021/talks/garlick.html Nice! What kind of feedback did you get? Is the FenICS community generally aware of reproducible deployment issues? It=E2=80=99s great to reach out to the numerical simulation community. Reproducible deployment and numerical simulation are two links in the long chain of reproducible science that we have to connect. Cheers, Ludo=E2=80=99. From MAILER-DAEMON Sat Apr 03 10:36:52 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lShOO-0002Yc-A7 for mharc-guix-science@gnu.org; Sat, 03 Apr 2021 10:36:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53024) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lShOM-0002Xy-FJ for guix-science@gnu.org; Sat, 03 Apr 2021 10:36:50 -0400 Received: from mail-out-auth2.hosts.co.uk ([212.84.127.1]:57020) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lShOK-0007ZY-NH for guix-science@gnu.org; Sat, 03 Apr 2021 10:36:50 -0400 Received: from maikeh336.claranet.co.uk ([79.123.23.187] helo=pancake.local) by smtp.hosts.co.uk with esmtpsa (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim) (envelope-from ) id 1lShOF-0004D6-Ea; Sat, 03 Apr 2021 15:36:43 +0100 Message-ID: Subject: Re: reproducibility of numerical experiments From: Paul Garlick To: Ludovic =?ISO-8859-1?Q?Court=E8s?= Cc: guix-science@gnu.org Date: Sat, 03 Apr 2021 15:36:40 +0100 In-Reply-To: <87mtug203e.fsf@gnu.org> References: <1cb914b16ee0dad6bf5102bb4b66fda3b020dbec.camel@tourbillion-technology.com> <87mtug203e.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: none client-ip=212.84.127.1; envelope-from=pgarlick@tourbillion-technology.com; helo=mail-out-auth2.hosts.co.uk X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 14:36:50 -0000 Hi Ludo, On Fri, 2021-04-02 at 17:11 +0200, Ludovic Courtès wrote: > Nice! What kind of feedback did you get? The feedback was very positive. Owing to the complexity of the software dependencies. many people in the finite element community would like a more dependable method of managing their installations. There were two questions on the day: i) Q: what happens if a repository disappears? Is the environment still reproducible? A: Yes. The Software Heritage Project provides a backup of all Guix packages. This is automatically used as a fallback, if needed. ii) Q: Suppose I am an end user who wishes to include some third-party python package, do I require it to exist as a Guix package to be able to have the "exact" reproducibility you described? A: There is a useful feature that allows a local channel to be defined. One can keep extra packages and one’s own solvers in the local channel. These are not visible in the main Guix repository but have all the same features regarding reproducibility. > Is the FenICS community generally aware of reproducible deployment > issues? Yes, reproducible deployment is seen as desirable, though difficult to achieve with the currently-used tools. Typically model development starts on a local workstation and is scaled-up to an HPC system when needed. Tools such as singularity are often used. However, the local and remote installations may differ and as a consequence applications that run on the local system may not run on the remote system. > It’s great to reach out to the numerical simulation community. > Reproducible deployment and numerical simulation are two links in the > long chain of reproducible science that we have to connect. Sure. My view is that Guix provides the machinery to transform what can be a stop-start mode of numerical model development to something that is more sustainable over time. Best regards, Paul. From MAILER-DAEMON Tue Apr 06 03:19:00 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lTfzH-00011t-UL for mharc-guix-science@gnu.org; Tue, 06 Apr 2021 03:18:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47718) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lTfzF-00011E-PG for guix-science@gnu.org; Tue, 06 Apr 2021 03:18:57 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:15077) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lTfzD-0003QK-Cs for guix-science@gnu.org; Tue, 06 Apr 2021 03:18:57 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AbhlmPaHHCO8dZCVnpLqF/5HXdLJzesId70hD?= =?us-ascii?q?6mlaTxtJfsuE0/2/hfhz73PJoRsyeFVlo9CPP6GcXWjRnKQf3aA9NaqvNTOIhE?= =?us-ascii?q?KGII1u5oPpwXnBNkTFmNJ1+rxnd8FFaeHYKFhhkILb7WCDc+oI5dmc7eSVgv3D?= =?us-ascii?q?xG1mVgFgZ8hbnmREIyOaCFAzeA5dGZEiHoGd7cYCnjy7ZB0sH6aGL10ke8SGmN?= =?us-ascii?q?HRjpLhZnc9dnsawS2Dlymh5rK/MxXw5GZ7bxp32rsp/WXIlAb0j5/T1M2T8RPX?= =?us-ascii?q?22/N45k+orKI9vJ5CNWImowpLFzX+2CVTb58UL6Pti1dmoySwWst+eOnnz4Qe+?= =?us-ascii?q?5o4zf6eGWtpBPs3BT7uQxekkPK+Bu9iXvnrdeRfkNDN+NxwaZQch7Y7A4LtNVk?= =?us-ascii?q?2KJE2AuixuFqJCKFuCz84tjOEztulkayvBMZ4JQupk0aS4sULL9dxLZvgX9oLA?= =?us-ascii?q?=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,308,1610406000"; d="scan'208";a="501676870" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Apr 2021 09:18:51 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Paul Garlick Cc: guix-science@gnu.org Subject: Re: reproducibility of numerical experiments References: <1cb914b16ee0dad6bf5102bb4b66fda3b020dbec.camel@tourbillion-technology.com> <87mtug203e.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 06 Apr 2021 09:18:50 +0200 In-Reply-To: (Paul Garlick's message of "Sat, 03 Apr 2021 15:36:40 +0100") Message-ID: <87pmz7q3s5.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2021 07:18:58 -0000 Hi Paul, Paul Garlick skribis: > There were two questions on the day: > > i) Q: what happens if a repository disappears? Is the environment > still reproducible? > A: Yes. The Software Heritage Project provides a backup of all Guix > packages. This is automatically used as a fallback, if needed. > > ii) Q: Suppose I am an end user who wishes to include some third-party > python package, do I require it to exist as a Guix package to be able > to have the "exact" reproducibility you described? > A: There is a useful feature that allows a local channel to be > defined. One can keep extra packages and one=E2=80=99s own solvers in the > local channel. These are not visible in the main Guix repository but > have all the same features regarding reproducibility. OK. >> Is the FenICS community generally aware of reproducible deployment >> issues? > > Yes, reproducible deployment is seen as desirable, though difficult to > achieve with the currently-used tools. Typically model development > starts on a local workstation and is scaled-up to an HPC system when > needed. Tools such as singularity are often used. However, the local > and remote installations may differ and as a consequence applications > that run on the local system may not run on the remote system. > >> It=E2=80=99s great to reach out to the numerical simulation community. >> Reproducible deployment and numerical simulation are two links in the >> long chain of reproducible science that we have to connect. > > Sure. My view is that Guix provides the machinery to transform what > can be a stop-start mode of numerical model development to something > that is more sustainable over time. Interesting, thank you! Ludo=E2=80=99. From MAILER-DAEMON Thu Apr 08 12:35:27 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lUXcs-0003GW-Ee for mharc-guix-science@gnu.org; Thu, 08 Apr 2021 12:35:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49178) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUXcq-0003Cb-12 for guix-science@gnu.org; Thu, 08 Apr 2021 12:35:24 -0400 Received: from korolev.univ-paris7.fr ([2001:660:3301:8000::1:2]:54466) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUXcn-0003CB-Q7 for guix-science@gnu.org; Thu, 08 Apr 2021 12:35:23 -0400 Received: from univ-paris7.fr (cgunivp7.sigu7.jussieu.fr [81.194.16.84]) by korolev.univ-paris7.fr (8.14.4/8.14.4/relay1/82085) with ESMTP id 138GZHJW019722 for ; Thu, 8 Apr 2021 18:35:17 +0200 Received: from [81.194.16.82] (account simon.tournier@paris7.jussieu.fr) by univ-paris7.fr (CommuniGate Pro WebUser 4.2.10) with HTTP id 247966652 for guix-science@gnu.org; Thu, 08 Apr 2021 18:35:17 +0200 From: "Simon Tournier" Subject: First French-speaking workshop on reproducible software environments To: guix-science@gnu.org X-Mailer: CommuniGate Pro WebUser Interface v.4.2.10 Date: Thu, 08 Apr 2021 18:35:17 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (korolev.univ-paris7.fr [194.254.61.138]); Thu, 08 Apr 2021 18:35:17 +0200 (CEST) X-Miltered: at korolev with ID 606F30C5.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 606F30C5.000 from cgunivp7.sigu7.jussieu.fr/cgunivp7.sigu7.jussieu.fr/null/univ-paris7.fr/ X-j-chkmail-Score: MSGID : 606F30C5.000 on korolev.univ-paris7.fr : j-chkmail score : . : R=. U=. O=. B=0.000 -> S=0.000 X-j-chkmail-Status: Ham Received-SPF: pass client-ip=2001:660:3301:8000::1:2; envelope-from=simon.tournier@paris7.jussieu.fr; helo=korolev.univ-paris7.fr X-Spam_score_int: -1 X-Spam_score: -0.2 X-Spam_bar: / X-Spam_report: (-0.2 / 5.0 requ) BAYES_05=-0.5, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2021 16:35:24 -0000 Hi, We are organizing the first French-speaking workshop on the reproducibility of software environments for scientists, engineers, and system administrators. The workshop will take place on-line on **May 17–18th** from 09:00 to 12:30 CEST. You can find details (in French) here: Thanks, simon From MAILER-DAEMON Thu Apr 08 20:57:21 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lUfSb-0004Sg-Qo for mharc-guix-science@gnu.org; Thu, 08 Apr 2021 20:57:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56100) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUdLd-0001xf-P1 for guix-science@gnu.org; Thu, 08 Apr 2021 18:42:02 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:33188) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUdLb-0005xt-Jq for guix-science@gnu.org; Thu, 08 Apr 2021 18:42:01 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 2344fa74; Thu, 8 Apr 2021 22:41:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:from:message-id; s=dkim; bh=5RAszca84H2Cpq568pojMKmVOcUDamBO4EzvCm2xIew=; b=ZqleVX+vyTEX yU+asRD1dvm9w+fbQZIMGl3Mir95hvvFeR341ec1z+yYE0nPwy8o29LSunrBGT/W P70z34z451YjJePA0jS7+T2IxP49hBtNxz1omAc7wFvluIlZnxjN7HA6A5SkqxTN Ug78YryfVXtdbsMdFJ2fTV/V4SJ/akgnnuC+nTJlnIxCl6qcHzMJmFXqZwx8x6Af oI7Ido9aL9yobui38FQjBYmchZLdEw2R5CLuGpHWJb29C2ora3pDGsKOOej0L9UK JRlHVjL30dgHjP0pQcXXSHp5S4A4GnWBepPFz+zPlxxXPY+rEoF1uu/E2adeRqbx qrICVe4mnQ== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 31b21105 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Thu, 8 Apr 2021 22:41:52 +0000 (UTC) Date: Thu, 08 Apr 2021 18:41:40 -0400 User-Agent: K-9 Mail for Android In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----J41BT3ZISW0A1N1BS6ESX5PQVVJD2S" Content-Transfer-Encoding: 7bit Subject: Re: First French-speaking workshop on reproducible software environments To: guix-science@gnu.org, Simon Tournier From: Julien Lepiller Message-ID: <70682EF7-1DEB-4E53-A025-63A110D713C1@lepiller.eu> Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Thu, 08 Apr 2021 20:57:21 -0400 X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2021 22:42:02 -0000 ------J41BT3ZISW0A1N1BS6ESX5PQVVJD2S Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable That's too early for me=2E Will the event be recorded? Le 8 avril 2021 12:35:17 GMT-04:00, Simon Tournier a =C3=A9crit : >Hi, > >We are organizing the first French-speaking workshop on the >reproducibility of software environments for scientists, engineers, and >system administrators=2E The workshop will take place on-line on **May >17=C2=9618th** from 09:00 to 12:30 CEST=2E You can find details (in Fren= ch) >here:=20 > > > > >Thanks, >simon ------J41BT3ZISW0A1N1BS6ESX5PQVVJD2S Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable That's too early for me=2E Will the event be recor= ded?

Le 8 avril 2021 12:35:17 GMT-04:00, = Simon Tournier <simon=2Etournier@univ-paris-diderot=2Efr> a =C3=A9cri= t :
Hi,

We are organizing the first French-speaki= ng workshop on the
reproducibility of software environments for scientis= ts, engineers, and
system administrators=2E The workshop will take plac= e on-line on **May
17=C2=9618th** from 09:00 to 12:30 CEST=2E You can f= ind details (in French)
here:

<https://hpc=2Eguix=2Einfo/even= ts/2021/atelier-reproductibilit=C3=A9-environnements/>


Th= anks,
simon


------J41BT3ZISW0A1N1BS6ESX5PQVVJD2S-- From MAILER-DAEMON Fri Apr 09 03:48:12 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lUlsC-0002AN-Gu for mharc-guix-science@gnu.org; Fri, 09 Apr 2021 03:48:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUlsB-0002AE-0j for guix-science@gnu.org; Fri, 09 Apr 2021 03:48:11 -0400 Received: from korolev.univ-paris7.fr ([2001:660:3301:8000::1:2]:56610) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUls8-0002mT-Hh for guix-science@gnu.org; Fri, 09 Apr 2021 03:48:10 -0400 Received: from univ-paris7.fr (cgunivp7.sigu7.jussieu.fr [81.194.16.84]) by korolev.univ-paris7.fr (8.14.4/8.14.4/relay1/82085) with ESMTP id 1397m42F021610; Fri, 9 Apr 2021 09:48:04 +0200 Received: from [81.194.16.82] (account simon.tournier@paris7.jussieu.fr) by univ-paris7.fr (CommuniGate Pro WebUser 4.2.10) with HTTP id 247983079; Fri, 09 Apr 2021 09:48:04 +0200 From: "Simon Tournier" Subject: Re: First French-speaking workshop on reproducible software environments To: Julien Lepiller , guix-science@gnu.org, Simon Tournier X-Mailer: CommuniGate Pro WebUser Interface v.4.2.10 Date: Fri, 09 Apr 2021 09:48:04 +0200 Message-ID: In-Reply-To: <70682EF7-1DEB-4E53-A025-63A110D713C1@lepiller.eu> References: <70682EF7-1DEB-4E53-A025-63A110D713C1@lepiller.eu> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (korolev.univ-paris7.fr [194.254.61.138]); Fri, 09 Apr 2021 09:48:04 +0200 (CEST) X-Miltered: at korolev with ID 607006B4.001 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 607006B4.001 from cgunivp7.sigu7.jussieu.fr/cgunivp7.sigu7.jussieu.fr/null/univ-paris7.fr/ X-j-chkmail-Score: MSGID : 607006B4.001 on korolev.univ-paris7.fr : j-chkmail score : . : R=. U=. O=. B=0.000 -> S=0.000 X-j-chkmail-Status: Ham Received-SPF: pass client-ip=2001:660:3301:8000::1:2; envelope-from=simon.tournier@paris7.jussieu.fr; helo=korolev.univ-paris7.fr X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2021 07:48:11 -0000 salut Julien, On Thu, 08 Apr 2021 18:41:40 -0400 Julien Lepiller wrote: >That's too early for me. Will the event be recorded? Well, we have not talked about it yet. The answer seems something like: we will try and then depending on the result. :-) Cheers, simon From MAILER-DAEMON Sat Apr 10 16:43:49 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lVKSK-0004wP-V9 for mharc-guix-science@gnu.org; Sat, 10 Apr 2021 16:43:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lVKSJ-0004wH-Fw for guix-science@gnu.org; Sat, 10 Apr 2021 16:43:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50738) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lVKSH-0005EP-4e; Sat, 10 Apr 2021 16:43:45 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=42016 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lVKSG-0002C2-Ms; Sat, 10 Apr 2021 16:43:44 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: =?utf-8?Q?S=C3=A9bastien?= Lerique Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> <87blb1wgna.fsf@eauchat.org> <87o8f1jee2.fsf@inria.fr> <87ft0bvqv5.fsf@eauchat.org> <87mtui1jxn.fsf@gnu.org> <87k0pmysyk.fsf@eauchat.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 21 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 10 Apr 2021 22:43:43 +0200 In-Reply-To: <87k0pmysyk.fsf@eauchat.org> (=?utf-8?Q?=22S=C3=A9bastien?= Lerique"'s message of "Thu, 01 Apr 2021 23:34:12 +0900") Message-ID: <87k0p927ls.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Apr 2021 20:43:47 -0000 Hi S=C3=A9bastien, S=C3=A9bastien Lerique skribis: > If, instead of resarting from scratch, I just run again `guix install > hello --no-offload`, I then get: > > guix install: error: got unexpected path > `/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bas= h:=20 > warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)' from > substituter I (re-)fixed this particular problem here: https://issues.guix.gnu.org/46362 So if you create a pack containing the =E2=80=98guix=E2=80=99 package from = current master, this problem should no longer be there. Ludo=E2=80=99. From MAILER-DAEMON Sun Apr 11 21:21:38 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lVlGk-0007ZQ-EH for mharc-guix-science@gnu.org; Sun, 11 Apr 2021 21:21:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43280) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lVlGj-0007ZD-Ap for guix-science@gnu.org; Sun, 11 Apr 2021 21:21:37 -0400 Received: from elegua.eauchat.org ([91.224.149.118]:34032 helo=eauchat.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lVlGh-0004eW-6F; Sun, 11 Apr 2021 21:21:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eauchat.org; s=mail; t=1618190487; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uNbzJ7e2dWlnnjCrS0WsCAtGHEQo59eFtOC+TxNCOEw=; b=wh3CmRDXW0FZUci8DwaM+P6cEbZbY/blBO2JvyvmFb64/Ql0HR2mXxadJyHv85cooBcQyM 2kW9rM57/jiO4HRYXAIPGioudpcLF4yQXsdYuZkfibXymI28aekOSgzIAU1H8dfGBmuDds D0dvqf6nYUvz0s6GljlLZy00/UujodE= References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> <87blb1wgna.fsf@eauchat.org> <87o8f1jee2.fsf@inria.fr> <87ft0bvqv5.fsf@eauchat.org> <87mtui1jxn.fsf@gnu.org> <87k0pmysyk.fsf@eauchat.org> <87k0p927ls.fsf@gnu.org> From: =?utf-8?Q?S=C3=A9bastien?= Lerique To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution In-reply-to: <87k0p927ls.fsf@gnu.org> Date: Mon, 12 Apr 2021 10:21:22 +0900 Message-ID: <874kgcz4a5.fsf@eauchat.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=91.224.149.118; envelope-from=sl@eauchat.org; helo=eauchat.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2021 01:21:37 -0000 Hi Ludo, On 11 Apr 2021 at 05:43, Ludovic Court=C3=A8s wrote: >> If, instead of resarting from scratch, I just run again `guix=20 >> install >> hello --no-offload`, I then get: >> >> guix install: error: got unexpected path >> `/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/ba= sh: >> warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)'=20 >> from >> substituter > > I (re-)fixed this particular problem here: > > https://issues.guix.gnu.org/46362 > > So if you create a pack containing the =E2=80=98guix=E2=80=99 package fro= m=20 > current > master, this problem should no longer be there. Success! With substitutes: $ guix install hello --no-offload [... installs ...] $ GUIX_PROFILE=3D"/home/s/sebastien-lerique/.guix-profile" $ . "$GUIX_PROFILE/etc/profile" $ hello Hello, world! Then my next steps are to try out a few more things, see if some=20 builds work (based on existing substitutes, not from scratch),=20 gather some interest here now that I can demo, and then come back=20 to explore builtin support for a non-root shared build daemon. Thank you!! S=C3=A9bastien From MAILER-DAEMON Mon Apr 12 08:43:22 2021 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1lVvuU-00008G-7B for mharc-guix-science@gnu.org; Mon, 12 Apr 2021 08:43:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41490) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lVvuR-00007x-9J for guix-science@gnu.org; Mon, 12 Apr 2021 08:43:19 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:10126) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lVvuN-0001lZ-Bp for guix-science@gnu.org; Mon, 12 Apr 2021 08:43:18 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3A8NhUgKhs839HrW9oD89td165sXBQXisji2hD?= =?us-ascii?q?6mlwRA09T+WznamV7Zcm/DXzjyscX2xlpMCYNMC7LU/02JZp7eAqXIuKcxLhvA?= =?us-ascii?q?KTRr1KzYyn+DH4Hj27y+g178ddWoxzEsf5A1Q/rcuS2mSFOvIhxNXCz6yyn+fZ?= =?us-ascii?q?yB5WIj1CUK1r4wdnBgvzKCQfLzVuPpY3GIGR4cBKvVObCBEqR/6mDXoIVfWrnb?= =?us-ascii?q?P2va/hCCR2ZSIP2U2rhTOs5KWSKWn94isj?= X-IronPort-AV: E=Sophos;i="5.82,216,1613430000"; d="scan'208";a="378414986" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Apr 2021 14:43:10 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: =?utf-8?Q?S=C3=A9bastien?= Lerique Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> <87blb1wgna.fsf@eauchat.org> <87o8f1jee2.fsf@inria.fr> <87ft0bvqv5.fsf@eauchat.org> <87mtui1jxn.fsf@gnu.org> <87k0pmysyk.fsf@eauchat.org> <87k0p927ls.fsf@gnu.org> <874kgcz4a5.fsf@eauchat.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 23 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 12 Apr 2021 14:43:09 +0200 In-Reply-To: <874kgcz4a5.fsf@eauchat.org> (=?utf-8?Q?=22S=C3=A9bastien?= Lerique"'s message of "Mon, 12 Apr 2021 10:21:22 +0900") Message-ID: <87im4rptb6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.104; envelope-from=ludovic.courtes@inria.fr; helo=mail3-relais-sop.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2021 12:43:19 -0000 Hi, S=C3=A9bastien Lerique skribis: > Success! With substitutes: > > $ guix install hello --no-offload > [... installs ...] > $ GUIX_PROFILE=3D"/home/s/sebastien-lerique/.guix-profile" > $ . "$GUIX_PROFILE/etc/profile" > $ hello > Hello, world! Yay! (Of course, =E2=80=9Cecho hello world=E2=80=9D would have been simple= r but not as as fun. :-)) > Then my next steps are to try out a few more things, see if some > builds work (based on existing substitutes, not from scratch),=20 > gather some interest here now that I can demo, and then come back to > explore builtin support for a non-root shared build daemon. Awesome, let us know how it goes! Ludo=E2=80=99.