[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Compress VCS checkouts?
From: |
Ludovic Courtès |
Subject: |
Re: Compress VCS checkouts? |
Date: |
Wed, 07 Feb 2018 22:56:28 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) |
Hi Eric,
Eric Bavier <address@hidden> skribis:
> On Mon, 05 Feb 2018 11:27:10 +0100
> address@hidden (Ludovic Courtès) wrote:
>
>> Hello!
>>
>> Eric Bavier <address@hidden> skribis:
>>
>> > A while back I toyed with the idea of a "Sources DVD" for Guix, much
>> > like Trisquel has. While looking into the feasability of this, I
>> > noticed that the git/svn/cvs/hg checkouts our packages use are not
>> > compressed in the store. So I wrote a simple script to compare the raw
>> > size of all source checkouts with their corresponding xz-compressed
>> > tarballs.
>> >
>> > At the time, I couldn't convince myself the trouble was worth the
>> > effort; we had only 385MiB of uncompressed source checkouts, which
>> > would compress to 135MiB.
>> >
>> > Since that initial investigation, the number of packages using VCS
>> > checkouts has risen from 50 to 389, and the on-disk size of that source
>> > has risen to 3784MiB uncompressed. We could compress that down to
>> > 538MiB.
>> >
>> > Compression in not free, but I think it would be nice for our build
>> > servers who have to store and serve (which, iirc, requires them to
>> > compress it anyhow) this source.
>>
>> ‘guix publish -C’ always compresses store items, unless they are regular
>> files ending in “.gz”, “.xz”, etc. So there’s no problem here.
>
> I see, ok.
>
>>
>> It seems to me that the only benefit of compressing checkouts would be
>> the use case you’re talking about. For everyday use, it’s actually more
>> efficient to leave the uncompressed, so that people don’t end up doing
>> round of compression/decompression when they run “guix build”.
>
> Often, people would end up fetching the compressed source from a
> substitute server, no?
Specifically, they download a compressed nar that contains the
uncompressed checkout.
> And currently I already need to uncompress what I get from a
> substitute server before building.
>
> But now I'm wondering why we bother with the "repack" portion of
> "patch-and-repack"?
Good question! It’s mostly because it “looked nicer” this way (and
maybe at the time some of the code did not properly handle non-tarball
sources.) I hear we could apply the same “reasoning” (if we can call it
this way) to checkouts. Hmm!
>> That said, at worst, this could be addressed by writing a specific tool
>> for the job.
>
> Yes. I was hoping it could be avoided though.
The tool could be relatively simple.
Ludo’.