Re: [Dragora-users] Proposal: add to qi possibility to switch type of tarlz compression granularity

[Antonio Diaz Diaz]

Hello,

DustDFG wrote:
> I am not sure that it is difficult to implement. We can just create
> variable tarlz_compression_granularity
> and and use it instead of --solid. I propose something like this:
>
> tarlz_compression_granularity="--solid" or
> tarlz_compression_granularity="--no-solid"

You may also lower the compression level to gain speed.

"--no-solid" should be the fastest option for files beyond a certain size, 
but it surely needs some experimentation on packages with lots of 
directories or small files because each directory requires a 512-byte header 
and each file takes at least 1024 bytes (uncompressed). This may cause a 
large size expansion on some packages. See in this example how a 3-byte file 
expands to 113 bytes and a directory takes 105 bytes (compressed):

$ echo foo > foo
$ mkdir bar
$ tarlz -9cf foo.tar.lz --no-solid foo bar
$ lzip -lvv foo.tar.lz
   dict   memb  trail   uncompressed     compressed   saved  name
   4 KiB     3      0           2560            262  89.77%  foo.tar.lz
 member      data_pos      data_size     member_pos    member_size
     1              0           1024              0            113
     2           1024            512            113            105
     3           1536           1024            218             44

So maybe the fastest compression option that prevents size expansion is the 
(tarlz's) default --bsolid at level -0:

  tarlz_compression_options="-0 --bsolid"

http://www.nongnu.org/lzip/manual/tarlz_manual.html#g_t_002d_002dbsolid
--bsolid
    When creating or appending to a compressed archive, use block 
compression. Tar members are compressed together in a lzip member until they 
approximate a target uncompressed size. The size can't be exact because each 
solidly compressed data block must contain an integer number of tar members. 
Block compression is the default because it improves compression ratio for 
archives with many files smaller than the block size.

Best regards,
Antonio.