Am 26.06.2017 um 12:08 hat Peter Lieven geschrieben:
Am 26.06.2017 um 11:57 schrieb Kevin Wolf:
Am 26.06.2017 um 11:20 hat Peter Lieven geschrieben:
So he chose a different algorithm (zstd). When I asked, he posted a
comparison of algorithms (however a generic one and not measured in the
context of qemu) that suggests that LZO would be slightly faster, but
have a considerable worse compression ratio with the settings that were
benchmarked.
My idea to choose LZO was that it is widely available and available in
any distro you can think of. We already have probing for it in configure.
My concern with ZSTD would be that it seems there are no packages
available for most distros and that it seems to be multi-threaded. I don't
know if this will cause any trouble?
The availability and that we already link against LZO is a good point. I
think we want to avoid a situation where compressed qcow2 files can't be
read by binaries of popular distributions - after all, downloadable
images are an important use case for compressed images.
As long as the default remains gzip I don't see any issues. If you choose
a different algorithm, you should know what you are doing.
If we add a better compression algorithm, then it's clear to me that the
goal is to make it the default in a year or two.
I think it's clear that if there is any serious interest in compression,
we'll want to support at least one more algorithm. What we still need to
evaluate is which one(s) to take, and whether a simple incompatible flag
in the header like in Den's patch is enough or whether we should add a
whole new header field for the compression algorithm (like we already
have for encryption).
From my side there clearly is interest in optimizing the compression. Its
even possible to speed up zlib by 3-4x times by choosing other parameters
for deflate which unfortunately are not compatible with our inflate settings.
I don't know if its worth creating a new header field. Even if we spent to bits
in the end (one for LZO and one for ZSDT). I think this wouldn't hurt. However,
there are likely to pop up new compression algorithms in the future and
a header would be more flexible.
That we could just use different parameters is an important hint that
maybe we don't just need bits to select an algorithm, but also some
space for parameters for the selected algorithm. In this case I think we
need a header change anyway.
The issue is, you don't know in advance how much space will be needed
for the paramters. Maybe it would be simpler to have different
settings for a compression algorithm threated as different algorithms.
Like zstd-fast, zstd, zstd-slow, gzip, lzo, lzo-slow etc.
Yes, how much space is needed for parameters depends on the algorithm.
Normally, when adding an incompatible feature, you would put the new
options inside the main header. However, nothing stops us from using a
header extension instead like we would with compatible features.
So I think this is what we could do:
* Add a header extension type per algorithm that contains the
parameters that are needed for this parameter
* Add a header field that selects the compression algorithm (e.g. a
32 bit field containing the matching header extension ID). Images
that contain compression header extensions that don't match this
header field (or images containing the same compression header
extension type more than once) are rejected
* Add an incompatible feature flag that indicates presence of the
header field + header extension