Re: [Qemu-block] [Qemu-devel] [PATCH 0/2] Improve qemu-img dd

[Eric Blake]

On 08/14/2018 09:56 PM, Eric Blake wrote:
> I was trying to test NBD fleecing by copying subsets of one
> file to another, and had the idea to use:
>
> $ export NBD drive to be fleeced on port 10809
> $ qemu-img create -f qcow2 copy $size
> $ qemu-nbd -f qcow2 -p 10810 copy
> $ qemu-img dd -f raw -O raw if=nbd://localhost:10809 of=nbd://localhost:10810 \
>      skip=$offset seek=$offset count=$((len/cluster)) bs=$cluster
>
> except that seek= wasn't implemented. And in implementing that,
> I learned that skip= is broken when combined with count=.
>
> [In the meantime, I had to use:
>
> $ export NBD drive to be fleeced on port 10809
> $ modprobe nbd
> $ qemu-nbd -c /dev/nbd0 -f raw nbd://localhost:10809
> $ qemu-nbd -c /dev/nbd1 -f qcow2 copy
> $ dd if=/dev/nbd0 of/dev/nbd1 \

Oops, left out one = on that line.

>      skip=$offset seek=$offset count=$((len/cluster)) bs=$cluster

And this needs to be Rather, skip=$((offset/cluster)) 
seek=$((offset/cluster)), unless iflag=skip_bytes oflag=seek_bytes is 
also in use.

What's more, it's essential to use conf=fdatasync when scripting this, 
otherwise, the dd process can end while the data is still in buffers, 
and if 'qemu-nbd -d /dev/nbd1' follows too closely, those buffers are 
lost instead of flushed.  (I lost the better part of a day figuring out 
why things worked when I did it by hand but not when I scripted it, 
until finally figuring out that the final flush is mandatory to avoid 
data loss).

> to get the behavior I needed (basically, create an empty qcow2
> destination file, then plug in the guest-visible data based on
> the subsets of the disk of my choosing, by reading the block
> status/dirty bitmap over NBD).  But bouncing through three
> NBD client/server pairs just so I can use plain 'dd' instead
> of just two pairs with 'qemu-img dd' feels dirty.
> ]
>
> Eric Blake (2):
>    qemu-img: Fix dd with skip= and count=
>    qemu-img: Add dd seek= option
>
>   qemu-img.c                 |  76 ++++++----
>   tests/qemu-iotests/160     |  15 +-
>   tests/qemu-iotests/160.out | 344 ++++++++++++++++++++++++++++++++++++++++++++-
>   3 files changed, 397 insertions(+), 38 deletions(-)

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org