[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parallel downloads
From: |
Leo Famulari |
Subject: |
Re: Parallel downloads |
Date: |
Wed, 13 Nov 2019 13:25:55 -0500 |
User-agent: |
Mutt/1.12.2 (2019-09-21) |
On Wed, Nov 13, 2019 at 11:16:53AM -0500, Mark H Weaver wrote:
> For these reasons, I'm inclined to think that parallel downloads is the
> wrong approach. If a single download process is not making efficient
> use of the available bandwidth, I'd be more inclined to look carefully
> at why it's failing to do so. For example, I'm not sure if this is the
> case (and don't have time to look right now), but if the current code
> waits until a NAR has finished downloading before asking for the next
> one, that's an issue that could be fixed by use of HTTP pipelining,
> without multiplying the memory usage.
>
> What do you think?
I agree that parallel downloads is a kludge to work around the issue of
slow set-up and tear-down of our download code. Pipelining would help a
lot, and we could also profile the relevant Guile code to see if there
are any easy speedups.
This issue was actually discussed a year ago:
https://lists.gnu.org/archive/html/guix-devel/2018-11/msg00148.html
I'll quote Ludo's suggestion from then:
> I’d be in favor of a solution where ‘guix substitute’ is kept alive
> across substitutions (like what happens with ‘guix substitute --query’),
> which would allow it to keep connections alive and thus save the TLS
> handshake and a few extra round trips per download.
- Re: Parallel downloads, (continued)
- Re: Parallel downloads, Pierre Neidhardt, 2019/11/10
- Re: Parallel downloads, Ludovic Courtès, 2019/11/12
- Re: Parallel downloads, John Soo, 2019/11/12
- Re: Parallel downloads, zimoun, 2019/11/12
- Re: Parallel downloads, Efraim Flashner, 2019/11/13
- Re: Parallel downloads, zimoun, 2019/11/13
- Re: Parallel downloads, Leo Famulari, 2019/11/12
- Re: Parallel downloads, Ludovic Courtès, 2019/11/17
- Re: Parallel downloads, Mark H Weaver, 2019/11/13
- Re: Parallel downloads, Pierre Neidhardt, 2019/11/13
- Re: Parallel downloads,
Leo Famulari <=
- Re: Parallel downloads, Pierre Neidhardt, 2019/11/13
- Re: Parallel downloads, Ludovic Courtès, 2019/11/17
- Re: Parallel downloads, Bengt Richter, 2019/11/06