[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: When substitute download + decompression is CPU-bound
From: |
Nicolò Balzarotti |
Subject: |
Re: When substitute download + decompression is CPU-bound |
Date: |
Mon, 14 Dec 2020 23:59:54 +0100 |
Ludovic Courtès <ludo@gnu.org> writes:
> Hi Guix!
>
Hi Ludo
> Quick decompression bench:
I guess this benchmark follows the distri talk, doesn't it? :)
File size with zstd vs zstd -9 vs current lzip:
- 71M uc.nar.lz
- 87M uc.nar.zst-9
- 97M uc.nar.zst-default
> Where to go from here? Several options:
> 1. Since ci.guix.gnu.org still provides both gzip and lzip archives,
> ‘guix substitute’ could automatically pick one or the other
> depending on the CPU and bandwidth. Perhaps a simple trick would
> be to check the user/wall-clock time ratio and switch to gzip for
> subsequent downloads if that ratio is close to one. How well would
> that work?
I'm not sure using heuristics (i.e., guessing what should work better,
like in 1.) is the way to go, as temporary slowdowns to the network/cpu
will during the first download would affect the decision.
> 2. Use Zstd like all the cool kids since it seems to have a much
> higher decompression speed: <https://facebook.github.io/zstd/>.
> 630 MB/s on ungoogled-chromium on my laptop. Woow.
I know this means more work to do, but it seems to be the best
alternative. However, if we go that way, will we keep lzip substitutes?
The 20% difference in size between lzip/zstd would mean a lot with slow
(mobile) network connections.
Nicolò
- When substitute download + decompression is CPU-bound, Ludovic Courtès, 2020/12/14
- Re: When substitute download + decompression is CPU-bound, Julien Lepiller, 2020/12/14
- Re: When substitute download + decompression is CPU-bound,
Nicolò Balzarotti <=
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Nicolò Balzarotti, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Nicolò Balzarotti, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Ludovic Courtès, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2020/12/15
- Re: When substitute download + decompression is CPU-bound, Ludovic Courtès, 2020/12/18
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2020/12/18