Re: When substitute download + decompression is CPU-bound

[Pierre Neidhardt]

Hi Ludo!

Ludovic Courtès <ludo@gnu.org> writes:

>> On Guillaume's graph, the compression speed at the default level 3 is
>> about 110 MB/s, while at level 10 it's about 40 MB/s, which is
>> approximately the gzip speed.
>>
>> If server compression time does not matter, then I agree, level >= 10
>> would be a good option.
>>
>> What about zstd level 19 then?  It's as slow as lzip to compress, but
>> decompresses still blazingly fast, which is what we are trying to
>> achieve here, _while_ offering a compression ration in the ballpark of
>> lzip level 6 (but still not that of lzip level 9).
>
> We could do that.  I suppose a possible agenda would be:
>
>   1. Start providing zstd susbstitutes anytime.  However, most clients
>      will keep choosing lzip because it usually compresses better.
>
>   2. After the next release, stop providing lzip substitutes and provide
>      only gzip + zstd-19.
>
> This option has the advantage that it wouldn’t break any installation.

But why would we keep gzip since it offers no benefits compared to zstd?
It feels like continuing to carry a (huge) burden forever...

Besides, dropping Lzip seems like a step backward in my opinion.  Users
with lower bandwidth (or simply further away from Berlin) will be
impacted a lot.

I would opt for dropping gzip instead, only to keep zstd-19 and lzip-9
(possibly plzip-9 if we update the bindings).

> It’s not as nice as the ability to choose a download strategy, as we
> discussed earlier, but implementing that download strategy sounds
> tricky.

If the user can choose their favourite substitute compression, I believe
it's usually enough since they are the best judge of their bandwidth /
hardware requirements.

Wouldn't this simple enough?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

signature.asc
Description: PGP signature