Re: [EXTERNAL]Re: Modifying squashfs usage for `guix pack` for supporting singularity

[Josh Marshall]

Thanks for getting back to me on this.

For the options I mentioned in the email, and the one not yet tracked in email 
since it is only in IRC is "--no-recovery".  These are what stood out to me 
when reading the `mksquashfs` documentation ( 
https://github.com/plougher/squashfs-tools/blob/master/USAGE ) and looking at 
`guix/scripts/pack.scm'.  The knowledge you're saying about the `guix pack` 
options is not communicated by `guix pack --list-formats`, which yields

```
$ guix pack --list-formats
The supported formats for 'guix pack' are:

  tarball       Self-contained tarball, ready to run on another machine
  squashfs      Squashfs image suitable for Singularity
  docker        Tarball ready for 'docker load'
```

All this seems to be tangential to the fact that `guix pack --format=squashfs 
...` fails on CentOS7 for an undescribed reason:
```
$ guix pack --format=squashfs bash-minimal
substitute: 
/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: 
warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
substitute: 
/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: 
warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
   /gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv
   /gnu/store/84vj4wgqm46rws1v73h9gc3hr29xdrs6-singularity-environment.sh.drv
   /gnu/store/iwrrg2knk45h4fps88jqfg92ff1x44zi-profile.drv
The following profile hooks will be built:
   /gnu/store/dgh995fyb41gr3il8r2770j2hlf22nzv-fonts-dir.drv
   /gnu/store/h5dr8a6vpzrk80wrjz586k3axpz9bhqa-manual-database.drv
   /gnu/store/svyngf4v16q3j8lgsql508573mvc64my-info-dir.drv
   /gnu/store/svyydyprj7id6gfz6hsifaf7v4g02brj-ca-certificate-bundle.drv
building CA certificate bundle...
building fonts directory...
building directory of Info manuals...
building database for manual pages...
building /gnu/store/iwrrg2knk45h4fps88jqfg92ff1x44zi-profile.drv...
building 
/gnu/store/84vj4wgqm46rws1v73h9gc3hr29xdrs6-singularity-environment.sh.drv...
/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: 
warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: 
warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
building 
/gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv...
\builder for 
`/gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv' 
failed with exit code 1
build of 
/gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv failed
View build log at 
'/var/log/guix/drvs/sa/ar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv.bz2'.
guix pack: error: build of 
`/gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv' 
failed
```

According to the OCI description, Singularity's .sif format could be skipped, 
but at my work it is /the/ standard format.  It sounds like there should be in 
program changes to describe supported container formats, and which systems 
support which.  This is in addition to getting squashfs container format 
working on CentOS7, and applying some changes to more correctly use 
`mksquashfs`.

________________________________________
From: Ludovic Courtès <address@hidden>
Sent: Thursday, March 12, 2020 9:51 AM
To: Josh Marshall
Cc: address@hidden
Subject: [EXTERNAL]Re: Modifying squashfs usage for `guix pack` for supporting 
singularity

Hi Josh,

Apologies for the delay!  (In general, it’s probably safer to report
bugs to address@hidden because they’re tracked and are less likely to
fall through the cracks.)

Josh Marshall <address@hidden> skribis:

> I ran into some trouble today when trying to package an image for singularity 
> on CentOS7.  nxck suggested I go into the cdoe and try adding the "-noappend" 
> option to mksquashfs in 'guix/guix/scripts/pack.scm'.  While looking at it, 
> it seems that it performs a multistep process to modify the squashfs image, 
> which kind of goes against the design of the utility.  A SO page ( 
> https://unix.stackexchange.com/a/459372 ) suggested the full hierarchy be 
> created and then added in a single invocation.  Does anyone have any opinions 
> on such a change or useful insights?

[...]

> It also appears the `-mkfs-time 0`, `-all-time 0`, and `-UTC` arguments have 
> been missing.

Could you explain what error you got and what makes you think these
options were missing?

IME, the images produced by ‘guix pack -f squashfs’ work fine with
Singularity 2.x and 3.x.

> Docker and Singularity havebeen moving towards support of the Open Containers 
> Initative, which prompts the discussion of if the docker and sinfularity 
> (squashfs) pack options should be depricated in favor of this standard: 
> https://github.com/opencontainers/image-spec

In fact, ‘guix pack -f docker’ produces an OCI image, IIUC.

Singularity 3.x introduced the Singularity Image Format (SIF), which is
yet another format.  AIUI, SIF is the recommended format for Singularity
3.x, isn’t it?

Thanks,
Ludo’.
---

The information in this email, including attachments, may be confidential and 
is intended solely for the addressee(s). If you believe you received this email 
by mistake, please notify the sender by return email as soon as possible.

ar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv.bz2
Description: ar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv.bz2