Re: [Dragora-users] Proposal: add to qi possibility to switch type of tarlz compression granularity

[DustDFG]

Thank you very much!

On Fri, May 20, 2022 at 11:06 PM Matías Fonzo <selk@dragora.org> wrote:
>
>
> Thanks!, I implemented it to be available by "etc/qirc", we could also
> set the option for a certain recipe, the source can be found here (at
> the moment the mirror has not been updated, but it will be):
>
> https://dragora.mirror.garr.it/current/sources/qi-2.9-rc1.tar.lz
> https://dragora.mirror.garr.it/current/sources/qi-2.9-rc1.tar.lz.sha256
>
> El 2022-05-19 11:26, Antonio Diaz Diaz escribió:
> > Hello,
> >
> > DustDFG wrote:
> >> I am not sure that it is difficult to implement. We can just create
> >> variable tarlz_compression_granularity
> >> and and use it instead of --solid. I propose something like this:
> >>
> >> tarlz_compression_granularity="--solid" or
> >> tarlz_compression_granularity="--no-solid"
> >
> > You may also lower the compression level to gain speed.
> >
> > "--no-solid" should be the fastest option for files beyond a certain
> > size, but it surely needs some experimentation on packages with lots
> > of directories or small files because each directory requires a
> > 512-byte header and each file takes at least 1024 bytes
> > (uncompressed). This may cause a large size expansion on some
> > packages. See in this example how a 3-byte file expands to 113 bytes
> > and a directory takes 105 bytes (compressed):
> >
> > $ echo foo > foo
> > $ mkdir bar
> > $ tarlz -9cf foo.tar.lz --no-solid foo bar
> > $ lzip -lvv foo.tar.lz
> >    dict   memb  trail   uncompressed     compressed   saved  name
> >    4 KiB     3      0           2560            262  89.77%  foo.tar.lz
> >  member      data_pos      data_size     member_pos    member_size
> >      1              0           1024              0            113
> >      2           1024            512            113            105
> >      3           1536           1024            218             44
> >
> > So maybe the fastest compression option that prevents size expansion
> > is the (tarlz's) default --bsolid at level -0:
> >
> >   tarlz_compression_options="-0 --bsolid"
> >
> > http://www.nongnu.org/lzip/manual/tarlz_manual.html#g_t_002d_002dbsolid
> > --bsolid
> >     When creating or appending to a compressed archive, use block
> > compression. Tar members are compressed together in a lzip member
> > until they approximate a target uncompressed size. The size can't be
> > exact because each solidly compressed data block must contain an
> > integer number of tar members. Block compression is the default
> > because it improves compression ratio for archives with many files
> > smaller than the block size.
> >
> > Best regards,
> > Antonio.
>