Dropping gzip-compressed substitutes

[Maxim Cournoyer]

Hello Guix!

While migrating our file system to the newly installed SSD-backed
storage on Berlin, I noticed that the guix-publish cached nars use a lot
of space: 6.5 TiB for the gzip compressed nars alone, or about 30% of
the new storage capacity.

The new array capacity was recently added so that running 'guix gc' on
it would be able to complete in a reasonable amount of time (remember
that no substitutes from the Cuirass workers can be retrieved while the
big GC lock is on, which greatly hurts the performance of the CI),
allowing to keep the size of /gnu/store in check.

Its capacity was chosen based on the assumption that /gnu/store
compresses well with zstd, leading to a 1.5-2.0 factor capacity boost.
This assumption doesn't hold in case we are storing a large quantity of
already-compressed data such as the /var/cache/guix/publish compressed
nars.

>From a past survey of the use of gzip substitutes [0], the result was a
small percentage of our users were still relying on them (1%).

Here's what I would suggest for a migration path:

1.  Promptly set up both a blog post and a NEWS entry announcing the
support for gzip substitutes is about to be phased out from the build
farm (1 month notice), urging users to upgrade their daemon to a version
>= 1.1.0.

2. Do the following in the maintenance repo, which should stop
advertising/generating new gzip substitutes:

--8<---------------cut here---------------start------------->8---
modified   hydra/modules/sysadmin/services.scm
@@ -580,9 +580,7 @@ to a selected directory.")
                    ;; 
<https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00097.html>
                    ;; for the compression ratio/decompression speed
                    ;; tradeoffs.
-                   ;; TODO: Eventually, disable gzip, as discussed at
-                   ;; 
<https://lists.gnu.org/archive/html/guix-devel/2021-03/msg00333.html>.
-                   (compression '(("gzip" 9) ("lzip" 9) ("zstd" 19)))
+                   (compression '(("lzip" 9) ("zstd" 19)))
                    (cache-bypass-threshold cache-bypass-threshold)
                    (workers publish-workers)))
--8<---------------cut here---------------end--------------->8---
 
3. Come up with a Guile script that is able to

  a) Strip gzip-related metadata from the .narinfo guix-publish metadata
  files
  b) recompute and update their 'Signature' field.

4. Finally, 'rm -r /var/guix/publish/gzip' and free about 6.5 TiB of data.

I'd like to move promptly with this as I'm expecting it may be necessary
to fully migrate the current file system to the new array (and reap the
benefits).

Thank you,

Maxim

[0]  https://lists.gnu.org/archive/html/guix-devel/2021-03/msg00333.html