qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2 1/4] qcow2: introduce compression type feature


From: Denis Plotnikov
Subject: Re: [PATCH v2 1/4] qcow2: introduce compression type feature
Date: Mon, 2 Mar 2020 14:29:31 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1



On 02.03.2020 14:24, Vladimir Sementsov-Ogievskiy wrote:
02.03.2020 11:21, Denis Plotnikov wrote:
The patch adds some preparation parts for incompatible compression type
feature to qcow2 allowing the use different compression methods for
image clusters (de)compressing.

It is implied that the compression type is set on the image creation and
can be changed only later by image conversion, thus compression type
defines the only compression algorithm used for the image, and thus,
for all image clusters.

The goal of the feature is to add support of other compression methods
to qcow2. For example, ZSTD which is more effective on compression than ZLIB.

The default compression is ZLIB. Images created with ZLIB compression type
are backward compatible with older qemu versions.

Adding of the compression type breaks a number of tests because now the
compression type is reported on image creation and there are some changes
in the qcow2 header in size and offsets.

The tests are fixed in the following ways:
     * filter out compression_type for all the tests
     * fix header size, feature table size and backing file offset
       affected tests: 031, 036, 061, 080
       header_size +=8: 1 byte compression type
                        7 bytes padding
       feature_table += 48: incompatible feture compression type
       backing_file_offset += 56 (8 + 48 -> header_change + fature_table_change)      * add "compression type" for test output matching when it isn't filtered
       affected tests: 049, 060, 061, 065, 144, 182, 242, 255

Signed-off-by: Denis Plotnikov <address@hidden>

I'm almost OK with this patch. Some notes below and:

Seems, new option should be handled in qcow2_amend_options among other unsupported ones (otherwise qcow2_amend_options aborts).

---
  qapi/block-core.json             |  22 ++++++-
  block/qcow2.h                    |  18 ++++-
  include/block/block_int.h        |   1 +
  block/qcow2.c                    | 109 +++++++++++++++++++++++++++++++
  tests/qemu-iotests/031.out       |  14 ++--
  tests/qemu-iotests/036.out       |   4 +-
  tests/qemu-iotests/049.out       | 102 ++++++++++++++---------------
  tests/qemu-iotests/060.out       |   1 +
  tests/qemu-iotests/061.out       |  34 ++++++----
  tests/qemu-iotests/065           |  20 +++---
  tests/qemu-iotests/080           |   2 +-
  tests/qemu-iotests/144.out       |   4 +-
  tests/qemu-iotests/182.out       |   2 +-
  tests/qemu-iotests/242.out       |   5 ++
  tests/qemu-iotests/255.out       |   8 +--
  tests/qemu-iotests/common.filter |   3 +-
  16 files changed, 255 insertions(+), 94 deletions(-)


[..]

--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -146,6 +146,12 @@ typedef struct QCowHeader {
        uint32_t refcount_order;
      uint32_t header_length;
+
+    /* Additional fields */
+    uint8_t  compression_type;
+
+    /* header must be a multiple of 8 */
+    uint8_t  padding[7];
  } QEMU_PACKED QCowHeader;
    typedef struct QEMU_PACKED QCowSnapshotHeader {
@@ -216,13 +222,16 @@ enum {
      QCOW2_INCOMPAT_DIRTY_BITNR      = 0,
      QCOW2_INCOMPAT_CORRUPT_BITNR    = 1,
      QCOW2_INCOMPAT_DATA_FILE_BITNR  = 2,
+    QCOW2_INCOMPAT_COMPRESSION_BITNR= 3,

checkpatch complains. I think, you can just use one space before '=' and don't
care about alignment.
ok

      QCOW2_INCOMPAT_DIRTY            = 1 << QCOW2_INCOMPAT_DIRTY_BITNR,
      QCOW2_INCOMPAT_CORRUPT          = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR,       QCOW2_INCOMPAT_DATA_FILE        = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, +    QCOW2_INCOMPAT_COMPRESSION      = 1 << QCOW2_INCOMPAT_COMPRESSION_BITNR,
        QCOW2_INCOMPAT_MASK             = QCOW2_INCOMPAT_DIRTY
                                      | QCOW2_INCOMPAT_CORRUPT
-                                    | QCOW2_INCOMPAT_DATA_FILE,
+                                    | QCOW2_INCOMPAT_DATA_FILE
+                                    | QCOW2_INCOMPAT_COMPRESSION,
  };
    /* Compatible feature bits */
@@ -369,6 +378,13 @@ typedef struct BDRVQcow2State {
        bool metadata_preallocation_checked;
      bool metadata_preallocation;
+    /*
+     * Compression type used for the image. Default: 0 - ZLIB
+     * The image compression type is set on image creation.
+     * The only way to change the compression type is to convert the image
+     * with the desired compression type set
+     */
+    Qcow2CompressionType compression_type;
  } BDRVQcow2State;
    typedef struct Qcow2COWRegion {
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6f9fd5e20e..2c6bb9dc99 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -57,6 +57,7 @@
  #define BLOCK_OPT_REFCOUNT_BITS     "refcount_bits"
  #define BLOCK_OPT_DATA_FILE         "data_file"
  #define BLOCK_OPT_DATA_FILE_RAW     "data_file_raw"
+#define BLOCK_OPT_COMPRESSION_TYPE  "compression_type"
    #define BLOCK_PROBE_BUF_SIZE        512
  diff --git a/block/qcow2.c b/block/qcow2.c
index 3c754f616b..fc5232a5d6 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1242,6 +1242,50 @@ static int qcow2_update_options(BlockDriverState *bs, QDict *options,
      return ret;
  }
  +static int validate_compression_type(BDRVQcow2State *s, Error **errp)
+{
+    /*
+     * Sanity check
+     * according to qcow2 spec, the compression type is 1-byte field
+     * but in BDRVQcow2State the compression_type is enum sizeof(int)
+     * so, the max compression_type value is 255.
+     */
+    if (s->compression_type > 0xff) {

This code is unreachable, I'd prefer assertion.
Hm, it shouldn't be here. I'll remove it.


+        error_setg(errp, "qcow2: compression type value is too big");
+        return -EINVAL;
+    }
+
+    switch (s->compression_type) {
+    case QCOW2_COMPRESSION_TYPE_ZLIB:
+        break;
+
+    default:
+        error_setg(errp, "qcow2: unknown compression type: %u",
+                   s->compression_type);
+        return -ENOTSUP;
+    }
+
+    /*
+     * if the compression type differs from QCOW2_COMPRESSION_TYPE_ZLIB
+     * the incompatible feature flag must be set
+     */
+    if (s->compression_type == QCOW2_COMPRESSION_TYPE_ZLIB) {
+        if (s->incompatible_features & QCOW2_INCOMPAT_COMPRESSION) {
+            error_setg(errp, "qcow2: Compression type incompatible feature "
+                             "bit must not be set");
+            return -EINVAL;
+        }
+    } else {
+        if (!(s->incompatible_features & QCOW2_INCOMPAT_COMPRESSION)) {
+            error_setg(errp, "qcow2: Compression type incompatible feature "
+                             "bit must be set");
+            return -EINVAL;
+        }
+    }
+
+    return 0;
+}
+


[..]

@@ -3379,6 +3453,27 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
          }
      }
  +    if (qcow2_opts->has_compression_type &&
+        qcow2_opts->compression_type != QCOW2_COMPRESSION_TYPE_ZLIB) {
+
+        ret = -EINVAL;
+
+        if (version < 3) {
+            error_setg(errp, "Compression type is only supported with "

Hmm. "Non-zlib compression type is only.." would be a bit more honest :)
valid.  to be changed.

+                       "compatibility level 1.1 and above (use version=v3 or "
+                       "greater)");
+            goto out;
+        }
+
+        switch (qcow2_opts->compression_type) {
+        default:
+            error_setg(errp, "Unknown compression type");
+            goto out;
+        }
+
+        compression_type = qcow2_opts->compression_type;
+    }
+


[..]

--- a/tests/qemu-iotests/065
+++ b/tests/qemu-iotests/065
@@ -88,23 +88,25 @@ class TestQMP(TestImageInfoSpecific):
  class TestQCow2(TestQemuImgInfo):
      '''Testing a qcow2 version 2 image'''
      img_options = 'compat=0.10'
-    json_compare = { 'compat': '0.10', 'refcount-bits': 16 }
-    human_compare = [ 'compat: 0.10', 'refcount bits: 16' ]
+    json_compare = { 'compat': '0.10', 'refcount-bits': 16, 'compression-type': 'zlib' } +    human_compare = [ 'compat: 0.10', 'compression type: zlib', 'refcount bits: 16' ]

over-80 line (and several below).

    class TestQCow3NotLazy(TestQemuImgInfo):
      '''Testing a qcow2 version 3 image with lazy refcounts disabled'''
      img_options = 'compat=1.1,lazy_refcounts=off'
      json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
-                     'refcount-bits': 16, 'corrupt': False }
-    human_compare = [ 'compat: 1.1', 'lazy refcounts: false',
+                     'refcount-bits': 16, 'corrupt': False,
+                     'compression-type': 'zlib' }
+    human_compare = [ 'compat: 1.1', 'compression type: zlib', 'lazy refcounts: false',
                        'refcount bits: 16', 'corrupt: false' ]
    class TestQCow3Lazy(TestQemuImgInfo):
      '''Testing a qcow2 version 3 image with lazy refcounts enabled'''
      img_options = 'compat=1.1,lazy_refcounts=on'
      json_compare = { 'compat': '1.1', 'lazy-refcounts': True,
-                     'refcount-bits': 16, 'corrupt': False }
-    human_compare = [ 'compat: 1.1', 'lazy refcounts: true',
+                     'refcount-bits': 16, 'corrupt': False,
+                     'compression-type': 'zlib' }
+    human_compare = [ 'compat: 1.1', 'compression type: zlib', 'lazy refcounts: true',
                        'refcount bits: 16', 'corrupt: false' ]

[..]






reply via email to

[Prev in Thread] Current Thread [Next in Thread]