Re: Exporting qcow2 images as raw data from ova file with qemu-nbd

[Eric Blake]

On 6/22/20 5:21 PM, Nir Soffer wrote:
> I'm trying to export qcow2 images from ova format using qemu-nbd.
>
> I create 2 compressed qcow2 images, with different data:
>
> $ qemu-img info disk1.qcow2
> image: disk1.qcow2
> file format: qcow2
> virtual size: 200 MiB (209715200 bytes)
> disk size: 384 KiB
> ...
>
> $ qemu-img info disk2.qcow2
> image: disk2.qcow2
> file format: qcow2
> virtual size: 200 MiB (209715200 bytes)
> disk size: 384 KiB
> ...
>
> And packed them in a tar file. This is not a valid ova but good enough
> for this test:
>
> $ tar tvf vm.ova
> -rw-r--r-- nsoffer/nsoffer 454144 2020-06-22 21:34 disk1.qcow2
> -rw-r--r-- nsoffer/nsoffer 454144 2020-06-22 21:34 disk2.qcow2

Thanks for the reproduction steps, this helps me reproduce locally.

> To get info about the disks in ova file, we can use:
>
> $ python -c 'import tarfile; print(list({"name": m.name, "offset":
> m.offset_data, "size": m.size} for m in tarfile.open("vm.ova")))'
> [{'name': 'disk1.qcow2', 'offset': 512, 'size': 454144}, {'name':
> 'disk2.qcow2', 'offset': 455168, 'size': 454144}]
>
> First I tried the obvious:
>
> $ qemu-nbd --persistent --socket=/tmp/nbd.sock --read-only --offset=512 vm.ova
>
> And it works, but it exposes the qcow2 data. I want to raw data so I
> can upload the guest
> data to ovirt, where is may be converted to qcow2 format.

So you definitely need two layers of interpretation: one that reads from 
an offset within the tar file, another that reads guest-visible data 
from the qcow2 contents at that offset.  Both qemu and nbdkit can 
perform the first layer, but at the moment, nbdkit cannot perform the 
second (our thoughts along that line have been that if qemu can read 
qcow2 files, there's no need to duplicate that effort in nbdkit).  So we 
either need to come up with a single qemu-nbd command line that can do 
it all, or else a process chain where nbdkit serves the offset over a 
Unix socket, then qemu-nbd consumes that socket to server the 
guest-visible data of the qcow2 format.

> $ qemu-img info --output json "nbd+unix://?socket=/tmp/nbd.sock"
> {
>      "virtual-size": 209715200,
>      "filename": "nbd+unix://?socket=/tmp/nbd.sock",
>      "format": "qcow2",
>   ...
> }
>
> Looking in qemu manual and qapi/block-core.json, I could construct this command:
>
> $ qemu-nbd --persistent --socket=/tmp/nbd.sock --read-only
> 'json:{"driver": "qcow2", "file": {"driver": "raw", "offset": 512,
> "size": 454144, "file": {"driver": "file", "filename": "vm.ova"}}}'

Yep, that's getting qemu to perform both layers.  The pseudo-format 
json:{...} is a bit of a pain to write but accurate; it is also possible 
to write the same command using --image-opts:

qemu-nbd --persistent --socket=/tmp/nbd.sock --read-only --image-opts \
driver=qcow2,file.driver=raw,file.offset=512,file.size=454144,\
file.file.driver=file,file.file.filename=vm.ova

> And it works:
>
> $ qemu-img info --output json "nbd+unix://?socket=/tmp/nbd.sock"
> {
>      "virtual-size": 209715200,
>      "filename": "nbd+unix://?socket=/tmp/nbd.sock",
>      "format": "raw"
> }

> I wonder if this is the best way to stack a qcow2 driver on top of a
> raw driver exposing
> a range from a tar file.

Yes, that looks best to me. As I said above, the only other option I can 
think of is to mix qemu-nbd and nbdkit, as in:

nbdkit -r -U - --filter=offset file vm.ova offset=512 range=454144 \
 --run 'qemu-nbd -k /tmp/nbd.sock -r -f qcow2 $uri'

which gives the same contents over /tmp/nbd.sock with a shorter command 
line but with more processes (and thus more computation spent per I/O) 
involved.

> I found similar example for gluster in:
> docs/system/device-url-syntax.rst.inc
>
> Richard suggested to try nbdkit tar plugin, but the plugin is not
> available on RHEL,
> and this adds additional dependency, when we already use qemu-nbd.

Rich just rewrote the tar plugin to use python instead of perl, which 
means it is that much easier for a future RHEL to pull it in.  We still 
ought to consider having a tar filter, either in place of or in addition 
to, the tar plugin (similar to how we recently converted nbdkit's ext4 
support from a plugin to a filter) - having a tar filter would allow you 
to read a compressed ova file (by combining the xz and tar filters to 
decompress then extract a file).  But right now, nbdkit doesn't support 
non-C filters (and given that our tar plugin was written first in perl 
and now in python, that still means translation to yet another language 
if the filter requires it to be in C).

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org