bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos

[Ludovic Courtès]

Hi again!

Until now hydra.gnu.org was using Hydra (the software) to serve not only
the Web interface but also all the .narinfo and /nar URLs (substitute
meta-data and substitutes).

Starting from now, hydra.gnu.org directs all .narinfo and corresponding
nar requests to ‘guix publish’ instead of Hydra.

‘guix publish’ should be faster and less resource-hungry than Hydra.  It
uses in-process gzip for nar compression instead of bzip2 (I chose level
7, which seems to provide compression ratios close to what bzip2
provides with its default compression level, while being 3 times
faster).  Unlike Hydra it never forks so for instance, 404 responses for
.narinfo URLs should be quicker.  Hopefully, that will improve the
worst-case (cache miss) throughput.

I configured nginx in such a way that the former Hydra-provided /nar
URLs (which are cached in nginx instances, in our
/var/guix/substitute/cache directories, etc.) are still available.
‘guix publish’ uses the /guix/nar URLs while Hydra uses /nar, so the
nginx config redirects to either Hydra or ‘guix publish’ depending on
the URL:

  
https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/nginx/hydra.gnu.org-locations.conf#n29

Hydra-provided .narinfos are still cached here and there; they’ll be
progressively expire and be replaced by ‘guix publish’-provided
.narinfos.

Let me know if you notice anything fishy!

Ludo’.