Re: [Qemu-devel] [PATCH RFC] external backup api

[John Snow]

On 02/06/2016 04:19 AM, Vladimir Sementsov-Ogievskiy wrote:
> On 05.02.2016 22:48, John Snow wrote:
>>
>> On 01/22/2016 12:07 PM, Vladimir Sementsov-Ogievskiy wrote:
>>> Hi all.
>>>
>>> This is the early begin of the series which aims to add external backup
>>> api. This is needed to allow backup software use our dirty bitmaps.
>>>
>>> Vmware and Parallels Cloud Server have this feature.
>>>
>> Have a link to the equivalent feature that VMWare exposes? (Or Parallels
>> Cloud Server) ... I'm curious about what the API there looks like.
> 
> For VMware you need their Virtual Disk Api Programming Guide
> http://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vddk60_programming.pdf
> 

Great, thanks!

> Look at Changed Block Tracking (CBT) , Backup and Restore.
> 
> For PCS here is part of SDK header, related to the topic:
> 
> ====================================
> /*
>  * Builds a map of the disk contents changes between 2 PITs.
>    Parameters
>    hDisk :       A handle of type PHT_VIRTUAL_DISK identifying
>                  the virtual disk.
>    sPit1Uuid :   Uuid of the older PIT.
>    sPit2Uuid :   Uuid of the later PIT.
>    phMap :       A pointer to a variable which receives the
>          result (a handle of type PHT_VIRTUAL_DISK_MAP).
>    Returns
>    PRL_RESULT.
> */
> PRL_METHOD_DECL( PARALLELS_API_VER_5,
>                 PrlDisk_GetChangesMap_Local, (
>         PRL_HANDLE hDisk,
>         PRL_CONST_STR sPit1Uuid,
>         PRL_CONST_STR sPit2Uuid,
>         PRL_HANDLE_PTR phMap) );
> 

Effectively giving you a dirty bitmap diff between two snapshots.
Something we don't currently genuinely support in QEMU.

> /*
>  * Reports the number of significant bits in the map.
>    Parameters
>    hMap :        A handle of type PHT_VIRTUAL_DISK_MAP identifying
>                  the changes map.
>    phSize :      A pointer to a variable which receives the
>          result.
>    Returns
>    PRL_RESULT.
> */
> PRL_METHOD_DECL( PARALLELS_API_VER_5,
>                 PrlDiskMap_GetSize, (
>         PRL_HANDLE hMap,
>         PRL_UINT32_PTR pnSize) );
> 

I assume this is roughly the dirty bit count, for us, this would be
dirty clusters. (Or whatever granularity you specified, but usually
clusters.)

> /*
>  * Reports the size (in bytes) of a block mapped by a single bit
>  * in the map.
>    Parameters
>    hMap :        A handle of type PHT_VIRTUAL_DISK_MAP identifying
>                  the changes map.
>    phSize :      A pointer to a variable which receives the
>          result.
>    Returns
>    PRL_RESULT.
> */
> PRL_METHOD_DECL( PARALLELS_API_VER_5,
>                 PrlDiskMap_GetGranularity, (
>         PRL_HANDLE hMap,
>         PRL_UINT32_PTR pnSize) );
> 

Basically a granularity query.

> /*
>  * Returns bits from the blocks map.
>    Parameters
>    hMap :        A handle of type PHT_VIRTUAL_DISK_MAP identifying
>                  the changes map.
>    pBuffer :     A pointer to a store.
>    pnCapacity :  A pointer to a variable holding the size
>          of the buffer and receiving the number of
>          bytes actually written.
>    Returns
>    PRL_RESULT.
> */
> PRL_METHOD_DECL( PARALLELS_API_VER_5,
>                 PrlDiskMap_Read, (
>         PRL_HANDLE hMap,
>         PRL_VOID_PTR pBuffer,
>         PRL_UINT32_PTR pnCapacity) );
> 

And this would be a direct bitmap query.

Is the expected usage here that the third party client will use this
bitmap to read the source image? Or do you query for the data from API?

I think the thought among block devs would be to opt for more of the
second option, and less allowing clients to directly interface with the
image files.

> =======================================
> 
> 
>>
>>> There is only one patch here, about querying dirty bitmap from qemu by
>>> qmp command. It is just an updated and clipped (hmp command removed) old
>>> my patch "[PATCH RFC v3 01/14] qmp: add query-block-dirty-bitmap".
>>>
>>> Before writing the whole thing I'd like to discuss the details. Or, may
>>> be there are existing plans on this topic, or may be someone already
>>> works on it?
>>>
>>> I see it like this:
>>>
>>> =====
>>>
>>> - add qmp commands for dirty-bitmap functions: create_successor,
>>> abdicate,
>>> reclaime.
>> Hm, why do we need such low-level control over splitting and merging
>> bitmaps from an external client?
>>
>>> - make create-successor command transaction-able
>>> - add query-block-dirty-bitmap qmp command
>>>
>>> then, external backup:
>>>
>>> qmp transaction {
>>>      external-snapshot
>>>      bitmap-create-successor
>>> }
>>>
>>> qmp query frozen bitmap, not acquiring aio context.
>>>
>>> do external backup, using snapshot and bitmap
>>>
>>> if (success backup)
>>>      qmp bitmap-abdicate
>>> else
>>>      qmp bitmap-reclaime
>>>
>>> qmp merge snapshot
>>> =====
>>>
>> Hm, I see -- so you're hoping to manage the backup *entirely*
>> externally, so you want to be able to reach inside of QEMU and control
>> some status conditions to guarantee it'll be safe.
>>
>> I'm not convinced QEMU can guarantee such things -- due to various flush
>> properties, race conditions on write, etc. QEMU handles all of this
>> internally in a non-public way at the moment.
> 
> Hm, can you be more concrete? What operations are dangerous? We can do
> them in paused state for example.
> 

I suppose if you're going to pause the VM, then it should be reasonably
safe, but recently there have been endeavors to augment the .qcow2
format to prohibit concurrent access, which might include a paused VM as
well, I'm not clear on the implementation.

If you do it via paused only, then you also don't need to expose the
freeze/rollback mechanisms: the existing clear mechanism alone is
sufficient:

(A) The frozen backup fails. Nothing new has been written, so we don't
need to adjust anything, we can just try again.
(B) The frozen backup succeeds. We can just clear the bitmap before
unfreezing.

I definitely have reservations about using this as a live fleecing
mechanism -- the backup block job uses a write-notifier to make
just-in-time backups of data before it is altered, leaving it the only
"safe" live backup mechanism in QEMU currently. (Alongside mirror.)

I actually have some patches from Fam to introduce a live fleecing
mechanism into QEMU (The idea being you create a point-in-time drive you
can get data from via NBD, then delete it when done) that might be more
appropriate, but I ran into a lot of problems with the patch. I'll post
the WIP for that patch to try to solicit comments on the best way forward.

Otherwise, My biggest question here is:
"What does fleecing a backup externally provide as a benefit over
backing up to an NBD target?"

You can already today perform incremental backups to an NBD target to
copy the data out via an external mechanism, is this not sufficient for
Parallels? If not, why?

>>
>>> In the following patch query-bitmap acquires aio context. This must be
>>> ofcourse dropped for frozen bitmap.
>>> But to make it in true way, I think, I should check somehow that this is
>>> not just frozen bitmap, but the bitmap frozen by qmp command, to avoid
>>> incorrect quering of bitmap frozen by internal backup (or other
>>> mechanizm).. May be, it is not necessary.
>>>
>>>
>>>
>>
>>
>