Re: [Qemu-devel] [PATCH] No change in userland tools after resizing qcow2 image

[John Snow]

On 07/19/2015 05:24 AM, Taeha Kim wrote:
> Hello,
> 
> 
> There is no change in userland tools after resizing qcow2 image except
> file utility.
> 
> For example when resize qcow2 image, the "file" utility is detectable
> increased size.
> However, the "ls", “stat”, and “du” utility still don't know how many
> change size of image is changed.
> The following patch enables to let userland tools - ls, du, stat -
> know and apply changed size after resizing qcow2 image created by the
> qemu-img tool.
> Currently, “file” utility can only know without this patch.
> 
> In addtion, "file" utility sometimes don't know qcow2 image format's
> version as follows.
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (Unknown)
> 
> 
> So, user can't have any information about qcow2 image size.
> 
> ======================
> Signed-off-by: Taeha Kim <address@hidden>
> diff --git a/block.c b/block.c
> index d088ee0..93427f8 100644
> --- a/block.c
> +++ b/block.c
> @@ -2529,6 +2529,9 @@ int bdrv_truncate(BlockDriverState *bs, int64_t offset)
>         if (bs->blk) {
>             blk_dev_resize_cb(bs->blk);
>         }
> +       if (ret == 0) {
> +           ret = truncate(bs->filename, offset);
> +       }
>     }
>     return ret;
> }
> =====================
> 
> 
> $ ./qemu/qemu-img --version
> qemu-img version 2.3.91, Copyright (c) 2004-2008 Fabrice Bellard
> 
> The how-to-reproduce is as follows with three reproduction.
> 
> First let’s say that we create a qcow2 image using qemu-img tool as
> follows. There is still no problem.
> 
> $ ./qemu/qemu-img create -f qcow2 -o preallocation=metadata 100G.qcow2 100G
> Formatting '100G.qcow2', fmt=qcow2 size=107374182400 encryption=off
> cluster_size=65536 preallocation='metadata' lazy_refcounts=off
> refcount_bits=16
> 
> $ ./qemu/qemu-img check 100G.qcow2
> No errors were found on the image.
> 1638400/1638400 = 100.00% allocated, 0.00% fragmented, 0.00% compressed 
> clusters
> Image end offset: 107390828544
> 
> $ ls -l 100G.qcow2
> -rw-r--r--.   1 devjames devjames 107390828544 Jul 15 09:55 100G.qcow2
> 
> $ stat 100G.qcow2
>   File: ‘100G.qcow2’
>   Size: 107390828544    Blocks: 32408          IO Block: 4096   regular file
> Device: fd00h/64768d    Inode: 5778245         Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> Context: unconfined_u:object_r:user_home_t:s0
> Access: 2015-07-15 09:55:17.269620129 +0900
> Modify: 2015-07-15 09:55:17.269620129 +0900
> Change: 2015-07-15 09:55:17.269620129 +0900
>  Birth: -
> 
> $ du -bb 100G.qcow2
> 107390828544    100G.qcow2
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (v3), 107374182400 bytes
> 
> But, from now on there is a problem.
> 
> Second let’s say we resize the qcow2 image size from 100G to 200G
> using current qemu-img without the patch.
> 
> $ ./qemu/qemu-img resize 100G.qcow2 200G
> Image resized.
> 
> $ ./qemu/qemu-img check 100G.qcow2
> No errors were found on the image.
> 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed 
> clusters
> Image end offset: 107390894080
> 
> $ ls -l 100G.qcow2
> -rw-r--r--. 1 devjames devjames 107390832128 Jul 15 10:02 100G.qcow2
> 
> $ stat 100G.qcow2
>   File: ‘100G.qcow2’
>   Size: 107390832128    Blocks: 32416          IO Block: 4096   regular file
> Device: fd00h/64768d    Inode: 5778245         Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> Context: unconfined_u:object_r:user_home_t:s0
> Access: 2015-07-15 10:02:04.842425479 +0900
> Modify: 2015-07-15 10:02:04.854425682 +0900
> Change: 2015-07-15 10:02:04.854425682 +0900
>  Birth: -
> 
> $ du -bb 100G.qcow2
> 107390832128    100G.qcow2
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
> 
> We can see that “ls”, “stat”, and “du” utilities don’t know change
> size of qcow2 image except “file” one.
> 
> Third let’s say we resize the qcow2 image size from 100G to 200G using
> qemu-img with the patch.
> 
> $ ./qemu/qemu-img resize 100G.qcow2 200G
> Image resized.
> 
> $ ./qemu/qemu-img check 100G.qcow2
> No errors were found on the image.
> 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed 
> clusters
> Image end offset: 107390894080
> 
> $ ls -l 100G.qcow2
> -rw-r--r--. 1 devjames devjames 214748364800 Jul 15 10:08 100G.qcow2
> 
> $ stat 100G.qcow2
>   File: ‘100G.qcow2’
>   Size: 214748364800    Blocks: 32416          IO Block: 4096   regular file
> Device: fd00h/64768d    Inode: 5778245         Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> Context: unconfined_u:object_r:user_home_t:s0
> Access: 2015-07-15 10:04:37.595018501 +0900
> Modify: 2015-07-15 10:08:01.622484709 +0900
> Change: 2015-07-15 10:08:01.622484709 +0900
>  Birth: -
> 
> $ du -bb 100G.qcow2
> 14748364800    100G.qcow2
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
> 
> Now we can see above all four utilities know change size of qcow2 image.
> So I made the patch above for the "qemu-img" tool so that "ls",
> “stat”, and “du” utility can be applied increased size of the qcow2
> image file.
> 

It seems to me that the real issue here is that the resize command you
are issuing doesn't respect your original pre-allocation desires.

When changing the size of the qcow2 file, it generally just updates the
headers and doesn't pre-allocate those storage blocks. The actual size
of the .qcow2 image doesn't take up disk space for empty blocks. That's
part of its design. The reason 'file' can tell the "virtual" size of the
disk is because it is reading the header to do so.

ls and friends are telling you the size of the actual file, as you've
noticed.

Changing the resize mechanisms to automatically truncate to achieve
pre-allocation is wrong, I'm afraid.

I think if anything, we'd want to allow the qemu-img tool to take
options for the resize, concerning pre-allocation strategy.

--js

> I have created a VM using qcow2 image after patching using this patch.
> The 100G.qcow2 has resized 200G is detected and available.
> $ qemu-kvm -smp 4 -m 4096 100G.qcow2 -cdrom
> ../iso/Fedora-Live-Workstation-x86_64-21-5.iso
> 
> Thanks.
> 
> 
> 
> Sincerely yours,
> Taeha Kim
> Senior Engineer, Network Functions Virtualization Part, Samsung Electronics
>