Re: [Qemu-discuss] [Qemu-devel] Incremental drive-backup with dirty bitmaps

[Eric Blake]

On 2/6/19 11:20 AM, Suman Swaroop wrote:
>   Hey, some continuation questions from above discussion,
> 

What above discussion?  Oh, you top-posted, so you mean the below
discussion.  (On technical lists, it's best to avoid top-posting, and to
instead reply inline to make the conversation easier to follow; it's
also okay to trim quoted parts irrelevant to your reply, as it can be
assumed that anyone joining the conversation can find the public list
archives to catch up on the full thread).

> 
>    1.
> 
>    Comments in blockdev-add command section in patch
>    https://patchwork.kernel.org/patch/9638133/ says that
>    “Note: This command is still a work in progress. It doesn't support all
>    block drivers among other things. Stay away from it unless you want to help
>    with its development.” Does it work with qcow2 and raw from version 2.3?

blockdev-add was declared stable in commit 79b7a77ed, v2.9 and beyond.
You are correct that for simpler uses, you could probably get raw and
qcow2 actions to work in 2.3, but working with older versions is more of
a downstream task so you may get less support figuring out how to target
older and newer versions simultaneously from this list.

Meanwhile, there have been enough other fixes with incremental backups
that you probably want to be using 3.1 or newer.  For that matter, a lot
of bitmap commands were still experimental in 3.1, but have been
converted to stable for the upcoming 4.0; depending on what you plan on
doing with dirty bitmaps, having to implement something twice to use
x-block-dirty-bitmap-merge from 3.1 and block-dirty-bitmap-merge from
4.0 can be a pain.

>    2.
> 
>    When new nodes are added in chain, there is an associated backing image
>    file as well. Some of these image files become redundant when a node is
>    merged with other nodes. While issuing qmp commands to qemu via libvirt tls
>    socket, there does not seem to be any functionality available to delete the
>    redundant image files. Basically when a node is streamed or committed, the
>    node and its backing image file can be deleted as they are not required
>    anymore. Is there any qmp command to achieve the same with the constraint
>    that we only have access to libvirt tls socket and cannot ssh to the host?

Trying to diagram what you are asking, to make sure I understand:

You stared with:

base <- image1 <- image2

and later did a block stream to get:

base <- image2 (contents of image1 now in image2)

or a block commit to get:

base <- image2 (contents of image1 now in base)

and are trying to figure out how to delete image1 from the file system,
now that it is no longer in use by the backing chain?  Why can't you
just 'rm image1'? Or if you are using libvirt to manage the storage pool
that image1 lives in, can't you use libvirt virStorage* APIs to remove
it?  Yes, it might be nice it the libvirt APIs for external snapshot
management had a bit more power to optionally auto-delete images that
are no longer in use - but that's more a question for the libvirt list
than this list.

Or are you asking about a node still in qemu's memory, that would show
up via the query-block command?

If it is something owned by qemu, you can always experiment with the
libvirt backdoor of 'virsh qemu-monitor-command' to send QMP commands to
qemu that libvirt has not yet coded up official support for; but it puts
you squarely in unsupported territory (if it works, great; if it breaks,
you get to keep both pieces).

In the meantime, I've cc'd you on my v3 posting of what I hope will
stabilize into the libvirt incremental backup APIs in time for libvirt
5.1 (I'm down to a couple of weeks, with still quite a few things to
shake out).

>    3.
> 
>    In filename path of blockdev-add command or target path of drive-backup
>    command can we provide nfs urld directly for the image path i.e
>    nfs://<ip>/foo(not a mounted directory)? This is related to the point 2
>    above that we do not have root ssh access to the host but only access to
>    its tls socket

The point of blockdev-add is that you should supply the parameters
needed by a particular driver.  An image path that can be short-cutted
to nfs:// for qemu-img also has a long form described by
BlockdevOptionsNfs (present since 2.9); so where you use:

{ "driver": "file", "filename": "/path/to/local" }

to get the file driver, you would instead use:

{ "driver": "nfs", "server": "<ip>", "path": "foo", ... }

for an NFS access.  If you are using libvirt to manage qemu, then
libvirt should be able to do all this on your behalf - except that
getting libvirt to use blockdev-add has been a multi-year project, and
Peter Krempa is still producing patches that will hopefully land in
libvirt 5.1 along those lines.  But again, libvirt questions may be
better asked on the libvirt list.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

signature.asc
Description: OpenPGP digital signature