Re: [PATCH] qcow2: Allow resize of images with internal snapshots

[Max Reitz]

On 22.04.20 22:53, Eric Blake wrote:
> We originally refused to allow resize of images with internal
> snapshots because the v2 image format did not require the tracking of
> snapshot size, making it impossible to safely revert to a snapshot
> with a different size than the current view of the image.  But the
> snapshot size tracking was rectified in v3, and our recent fixes to
> qemu-img amend (see 0a85af35) guarantee that we always have a valid
> snapshot size.  Thus, we no longer need to artificially limit image
> resizes, but it does become one more thing that would prevent a
> downgrade back to v2.  And now that we support different-sized
> snapshots, it's also easy to fix reverting to a snapshot to apply the
> new size.
> 
> Upgrade iotest 61 to cover this (we previously had NO coverage of
> refusal to resize while snapshots exist).  Note that the amend process
> can fail but still have effects: in particular, since we break things
> into upgrade, resize, downgrade, if a failure does not happen until a
> later phase (such as the downgrade attempt), earlier steps are still
> visible (a truncation and downgrade attempt will fail, but only after
> truncating data).  But even before this patch, an attempt to upgrade
> and resize would fail but only after changing the image to v3.  In
> some sense, partial image changes on failure are inevitible, since we
> can't avoid a mid-change EIO (if you are trying to amend more than one
> thing at once, but something fails, I hope you have a backup image).
> 
> Signed-off-by: Eric Blake <address@hidden>
> ---
>  block/qcow2-snapshot.c     |  21 ++++--
>  block/qcow2.c              |  31 ++++++--
>  tests/qemu-iotests/061     |  23 ++++++
>  tests/qemu-iotests/061.out | 144 +++++++++++++++++++++++++++++++++++++
>  4 files changed, 211 insertions(+), 8 deletions(-)
> 
> diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c
> index 82c32d4c9b08..3f9e48738d0b 100644
> --- a/block/qcow2-snapshot.c
> +++ b/block/qcow2-snapshot.c
> @@ -23,6 +23,7 @@
>   */
> 
>  #include "qemu/osdep.h"
> +#include "sysemu/block-backend.h"
>  #include "qapi/error.h"
>  #include "qcow2.h"
>  #include "qemu/bswap.h"
> @@ -775,10 +776,22 @@ int qcow2_snapshot_goto(BlockDriverState *bs, const 
> char *snapshot_id)
>      }
> 
>      if (sn->disk_size != bs->total_sectors * BDRV_SECTOR_SIZE) {
> -        error_report("qcow2: Loading snapshots with different disk "
> -            "size is not implemented");
> -        ret = -ENOTSUP;
> -        goto fail;
> +        BlockBackend *blk = blk_new(bdrv_get_aio_context(bs),
> +                                    BLK_PERM_RESIZE, BLK_PERM_ALL);
> +        ret = blk_insert_bs(blk, bs, &local_err);

I wonder whether maybe we should reintroduce blk_new_with_bs().

> +        if (ret < 0) {
> +            blk_unref(blk);
> +            error_report_err(local_err);
> +            goto fail;
> +        }
> +
> +        ret = blk_truncate(blk, sn->disk_size, true, PREALLOC_MODE_OFF,
> +                           &local_err);
> +        blk_unref(blk);
> +        if (ret < 0) {
> +            error_report_err(local_err);
> +            goto fail;
> +        }
>      }
> 
>      /*

Looks good.

> diff --git a/block/qcow2.c b/block/qcow2.c
> index b524b0c53f84..29047c33b7e5 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -3988,14 +3988,21 @@ static int coroutine_fn 
> qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
> 
>      qemu_co_mutex_lock(&s->lock);
> 
> -    /* cannot proceed if image has snapshots */
> -    if (s->nb_snapshots) {
> -        error_setg(errp, "Can't resize an image which has snapshots");
> +    /*
> +     * Even though we store snapshot size for all images, it was not
> +     * required until v3, so it is not safe to proceed for v2.
> +     */
> +    if (s->nb_snapshots && s->qcow_version < 3) {
> +        error_setg(errp, "Can't resize a v2 image which has snapshots");
>          ret = -ENOTSUP;
>          goto fail;
>      }
> 
> -    /* cannot proceed if image has bitmaps */
> +    /*
> +     * For now, it's easier to not proceed if image has bitmaps, even
> +     * though we could resize bitmaps, because it is not obvious
> +     * whether new bits should be set or clear.

The previous comment was incorrect as well, but actually
qcow2_truncate_bitmaps_check() doesn’t return an error when there is any
bitmap, but only if there are non-active bitmaps, or active bitmaps that
cannot be modified.  So for non-disabled bitmaps, we generally do
happily proceed.

> +     */
>      if (qcow2_truncate_bitmaps_check(bs, errp)) {
>          ret = -ENOTSUP;
>          goto fail;

[...]

> diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
> index ce285d308408..fdfb8fab5fb6 100755
> --- a/tests/qemu-iotests/061
> +++ b/tests/qemu-iotests/061
> @@ -111,6 +111,29 @@ $PYTHON qcow2.py "$TEST_IMG" dump-header
>  $QEMU_IO -c "read -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
>  _check_test_img
> 
> +echo
> +echo "=== Testing resize with snapshots ==="
> +echo
> +_make_test_img -o "compat=0.10" 32M
> +$QEMU_IO -c "write -P 0x2a 24M 64k" "$TEST_IMG" | _filter_qemu_io
> +$QEMU_IMG snapshot -c foo "$TEST_IMG"
> +$QEMU_IMG resize "$TEST_IMG" 64M                         # fails
> +$PYTHON qcow2.py "$TEST_IMG" dump-header

What am I looking for in the header dump?

> +$QEMU_IMG amend -o "compat=1.1,size=128M" "$TEST_IMG"    # succeeds
> +$PYTHON qcow2.py "$TEST_IMG" dump-header

The v2 -> v3 change?  I think a _img_info (| grep compat) would be
better suited (or just a “$QEMU_IMG amend ... || echo ERROR”).

(I don’t like changing that many dump-header outputs whenever the
header_length or the feature table length change for some reason.)

Also, I personally prefer self-testing tests, because I don’t trust
myself when I have to interpret the reference output on my own...  As
such, I think it would make sense to not just put those “# fails”
comments here, but an explicit test on $? instead.  (E.g. by
“|| echo ERROR”, although I can see that would be weird in the
expected-failure case as “&& echo ERROR”.)

> +$QEMU_IMG snapshot -c bar "$TEST_IMG"
> +$QEMU_IMG resize --shrink "$TEST_IMG" 64M                # succeeds
> +$PYTHON qcow2.py "$TEST_IMG" dump-header
> +$QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG"    # fails, image left 
> v3
> +$PYTHON qcow2.py "$TEST_IMG" dump-header

Again, a grep for the image size would help focus the reference output.

(In addition, _img_info would give us snapshot information.  Might be
interesting.  So maybe the best thing would be to grep the image
version, image size, and snapshot list from the image info every time.)

Speaking of the image size.  Is it intentional that the size is changed
to 32 MB?  Should amend work more like a transaction, in that we should
at least do a loose check on whether the options can be changed before
we touch the image?

Also, there’s a problem of ordering here.  The command as you’ve written
doesn’t have this specific problem, but imagine the size was still 128
MB before (just like the snapshot).  Then what the command does depends
on the order in which the operations are executed: If we change the
version first, the size cannot be changed because of the internal
snapshot.  If we change the size first, the version can no longer be
changed because the internal snapshot has a different size than the image.


(Hypothetical problems that turn out not to be any in practice:

Or, maybe more interesting: What if we try to amend to
compat=0.10,size=128M here?

If the size is changed first, the command will succeed, because then the
snapshot size matches the image size again.  If qemu-img attempts to
change the version first, the whole command will fail.

As I noted above, the size is indeed changed before the version (hence
us getting a 32 MB v3 image here), so the compat=0.10,size=128M amend
would indeed succeed.  But that’s luck.

OTOH, that means that if you have a v2 image with a snapshot and want to
make it a v3 image and resize it at the same time, that would fail by
the same logic, because the size cannot be changed for v2 images.  So
for that case it’d be bad luck.

But we always upgrade an image first in the amend process and downgrade
it last, to address specifically such cases: Allow adding new features
along with the upgrade, and strip unsupported features right before the
downgrade.  So that’s all good.  But I think it still shows that the
dependencies are getting a bit hairy.)

Max

signature.asc
Description: OpenPGP digital signature