[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Monotone-devel] Re: zlib vs gzip

From: Markus Schiltknecht
Subject: Re: [Monotone-devel] Re: zlib vs gzip
Date: Tue, 30 Oct 2007 13:48:45 +0100
User-agent: Mozilla-Thunderbird (X11/20071008)


Jack Lloyd wrote:
Yes with that assumption I'd have to agree, but on anything else (eg,
4-way Opterons, old Unix machines with poor CPU and relatively
powerful I/O, etc) it probably hurts quite a bit. Even on my Core2
(with a RAID-1 setup, so each write has to hit all three disks), it is
faster (wall clock) to write 200 Mb to disk than it is to first
compress it with gzip and write the 30 Mb compressed result to disk.

Hm.. wow, yeah, on a very quick test, I got ~ 170 MB/s plain throughput, but only 13.5 MB/s when piping through gzip. Is compression really that slow even on modern hardware? I thought that changed long ago, well...

Thanks for pointing that out.

I remember the LZMA benchmarks [1], those seem to be more representative than my quick test. They list the following throughputs for an AMD mobile Athlon XP2400+ (in MB/sec):

                compression      decompression     compressed size (%)
gzip -1             18                61                  40,6
gzip -6              8.3              63                  37.2
gzip -9              3.0              65                  37.0
bzip2 -9             1.3               5.3                34.0
lzmash -9          < 0.27             20                  25.4

Thus, not even gzip -1 compression is close to a harddrive's throughput. Only gzip decompression comes somewhat close, probably making it quite a good fit.

I'm reminded of TOAST in PostgreSQL. They use a fast variant of an LZ compression algorithm. I'm wondering what throughput that achieves.



[1]: some LZMA benchmarks:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]