Re: [Qemu-devel] [Bug 1399191] [NEW] Large VHDX image size

[Jeff Cody]

On Thu, Jan 22, 2015 at 05:20:20AM +0000, Lokesha, Amulya wrote:
>     
>

[...]
 
>     
>>   Yes, I have sent a patch that I believe fixes the issue (I cc'ed you on
>>   the patch).  If you wouldn't mind testing, and verifying that it fixes
>>   your particular issue, that would be great.  I tested on Windows Server
>>   2012 w/Hyper-V, and I was able to verify the original problem and the fix.
>>    
>>   On what the issue was:
>>    
>>   The v1.0.0 spec for VHDX denotes that the FileOffsetMB portion of the
>>   block state field is 'reserved' for blocks in the state
>>   PAYLOAD_BLOCK_ZERO.  Before, we just went ahead and wrote the file offset
>>   value for that block into the field, and just ignored it during reads.
>>    
>>   If we force the FileOffsetMB field to be zero, then Hyper-V is able to
>>   read the images.  Furthermore, inspecting images converted by Hyper-V
>>   shows that Hyper-V writes '0' for the FileOffsetMB field in BAT entries
>>   that are in the PAYLOAD_BLOCK_ZERO state.
>>    
>>   The patch I sent mimics that behavior, and forces any BAT writes of
>>   PAYLOAD_BLOCK_ZERO state to have a FileOffsetMB value of 0.
>     
>     
>    Hi Jeff,
>     
>    Thanks a lot for the fix. We tested the VHDX creation with the new patch
>    fix and were able to get it deployed successfully into our Windows HyperV
>    Server 2012.
>

You're welcome!
 
>    However we have one more issue and were hoping if we could get any help.
>    We have a vmdk image (size 13MB) with a 500 GB data disk size. After
>    conversion of vmdk image to vhdx format, the image size is 126 GB.
>     
>    # qemu-img info Test.vmdk
>    image: Test.vmdk
>    file format: vmdk
>    virtual size: 500G (536870912000 bytes)
>    disk size: 13M
>    cluster_size: 65536
>    Format specific information:
>        cid: 4124953209
>        parent cid: 4294967295
>        create type: streamOptimized
>        extents:
>            [0]:
>                compressed: true
>                virtual size: 536870912000
>                filename: Test.vmdk
>                cluster size: 65536
>                format:
>     
>    qemu conversion to vhdx done as below
>    # qemu-img convert -p -o subformat=dynamic -f vmdk -O vhdx Test.vmdk
>    Test.vhdx
>        (100.00/100%)
>     
>    # ls -ltrh
>    total 69M
>    -rw-r--r-- 1 root root  13M Jan 21 21:42 Test.vmdk
>    -rw-r--r-- 1 root root 126G Jan 22 05:03 Test.vhdx
>     
>    # qemu-img info Test.vhdx
>    image: Test.vhdx
>    file format: vhdx
>    virtual size: 500G (536870912000 bytes)
>    disk size: 56M
>    cluster_size: 33554432
>     
>     
>    When we tried to copy this to the SCVMM manager server (Windows OS) it
>    failed due to disk limitation. Windows does think it is a 126 GB file. Can
>    anything be done to fix this issue?
>     
>    Thanks,
>    Amulya

It sounds like a 126G sparse image was created (reported size 126G, on
disk size 55M), and QEMU still recognizes it as a 500G image (file
size sounds large, but nothing indicates yet the vhdx image isn't
valid, at least).

I am not sure what your disk limitation limit was under Windows, can
you elaborate?

I tried to reproduce it here, and could not:

# ./qemu-img create -f vmdk test-500G.vmdk 500G

# ./qemu-img convert -p -o subformat=dynamic -f vmdk -O vhdx test-
500G.vmdk test-500G.vhdx

# ls -lhs test-500G*
1.3M -rw-r--r--. 1 jcody jcody 8.0M Jan 22 10:25 test-500G.vhdx
136K -rw-r--r--. 1 jcody jcody  63M Jan 22 10:25 test-500G.vmdk

I was also able to inspect test.vhdx on Windows Server 2012, using
Hyper-V Manager, and it reported that it was a vhdx dynamic disk, with
a maximum size of 500G.

Can you place your source vmdk image someplace for me to download and
test with, if it does not contain sensitive data?  You can send me the
link offlist, if you like.

Thanks,
Jeff

>    > >
>    > >
>    > > You received this bug notification because you are subscribed to the
>    bug report.
>    > > [5]https://bugs.launchpad.net/bugs/1399191
>    > >


[...]

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1399191

Title:
  Large VHDX image size

Status in QEMU:
  New

Bug description:
  We are trying to convert a VMDK image to VHDX image for deploy to HyperV 
Server ( SCVMM 2012 SP1) using qemu-img.
  We tried converting the image using both 'fixed' as well as 'dynamic' format. 
We found that both the disks occupy the same size of 50GB. When the same is 
done with VHD image, we found that the dynamic disks are much lesser in size (5 
GB) than the fixed disk (50GB). 

  Why is that the VHDX generates large sized images for both the
  formats?

  The following commands were used to convert the vmdk image to VHDX
  format

  1. qemu-img convert -p -o subformat=fixed  -f vmdk -O vhdx Test.vmdk
  Test-fixed.vhdx

  qemu-img info Test-fixed.vhdx
  image: Test-fixed.vhdx
  file format: vhdx
  virtual size: 50G (53687091200 bytes)
  disk size: 50G
  cluster_size: 16777216


  
  2. qemu-img convert -p -o subformat=dynamic  -f vmdk -O vhdx Test.vmdk 
Test-dynamic.vhdx

  qemu-img info Test-dynamic.vhdx
  image: Test-dynamic.vhdx
  file format: vhdx
  virtual size: 50G (53687091200 bytes)
  disk size: 50G
  cluster_size: 16777216

  
  We tried this with the following version of qemu
  1. qemu-2.0.0
  2. qemu-2.1.2
  3. qemu-2.2.0-rc4

  
  Please let us know how to create compact VHDX images using qemu-img.
  Thank you

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1399191/+subscriptions