Re: When substitute download + decompression is CPU-bound

[Pierre Neidhardt]

Ludovic Courtès <ludo@gnu.org> writes:

> One takeaway for me is that zstd decompression remains an order of
> magnitude faster than the others, regardless of the compression level.
>
> Another one is that at level 10 and higher zstd achieves compression
> ratios that are more in the ballpark of lzip.

Hmmm, this is roughly true for lzip < level 6, but as soon as lzip hits level 6
(the default!) it compresses up to twice as much!

> If we are to change the compression methods used at ci.guix.gnu.org, we
> could use zstd >= 10.

On Guillaume's graph, the compression speed at the default level 3 is
about 110 MB/s, while at level 10 it's about 40 MB/s, which is
approximately the gzip speed.

If server compression time does not matter, then I agree, level >= 10
would be a good option.

What about zstd level 19 then?  It's as slow as lzip to compress, but
decompresses still blazingly fast, which is what we are trying to
achieve here, _while_ offering a compression ration in the ballpark of
lzip level 6 (but still not that of lzip level 9).

> We could also drop gzip, but there are probably pre-1.1 daemons out
> there that understand nothing but gzip¹, so perhaps that’ll have to
> wait.  Now, compressing substitutes three times may be somewhat
> unreasonable.

Agreed, maybe release an announcement and give it a few months / 1 year?

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

signature.asc
Description: PGP signature