[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Qemu and Changed Block Tracking

From: Peter Lieven
Subject: Re: [Qemu-devel] Qemu and Changed Block Tracking
Date: Thu, 23 Feb 2017 15:29:24 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1

Am 22.02.2017 um 22:17 schrieb John Snow:

On 02/22/2017 03:45 AM, Peter Lieven wrote:
Am 21.02.2017 um 22:13 schrieb John Snow:
On 02/21/2017 07:43 AM, Peter Lieven wrote:

is there anyone ever thought about implementing something like VMware
CBT in Qemu?



A bit outdated now, but:

and also a summary I wrote not too far back (PDF):

and I'm sure the Virtuozzo developers could chime in on this subject,
but basically we do have something similar in the works, as eblake says.
Hi John, Hi Erik,

thanks for your feedback. Are you both the ones working primary on this topic?
If there is anything to review or help needed, please let me know.

I've been working on incremental backups; Fam and I now co-maintain

Vladimir Sementsov-Ogievskiy has been working on bitmap persistence and
migration from Virtuozzo; as well as the NBD specification amendment to
allow us to fleece images with dirty bitmaps.

(Check the wiki and the whitepaper I linked!)

Eric has been guiding the review process for the NBD side of things.

My 2 cents:
I thing I had in mind if there is no image fleecing available, but fetching the 
dirty bitmap
from external would be a feauture to put a write lock on a block device.
Write lock means, drain all pending writes and queue all further writes until 
unlock (as if they
were throttled to zero). This could help fetch consistent backups from storage 
device (thinking of iSCSI SAN) without
the help of the hypervisor to actually transfer data (no load in the frontend 
network or the host). What would further
be needed is a write generation for each block, not just only a dirty bitmap.

In this case something like this via QMP (and external software) should work:
  gen =  write generation of last backup (or 0 for full backup)
  do {
      nextgen = fetch current write generation (via QMP)
As Eric said, there's a lot of hostility to using QMP as a metadata
transmission protocol.

      dirtymap = send all block whose write generation is greater than 'gen' 
(via QMP)
      dirtycnt = 0
      foreach block in dirtymap {
                copy to backup via external software
      gen = nextgen
  } while (dirtycnt < X)         <--- to achieve this a thorttling or similar 
might be needed

fsfreeze (optional)
write lock (via QMP)
backupgen = fetch current write generation (via QMP)
dirtymap = send all block whose write generation is greater than 'gen' (via QMP)
foreach block in dirtymap {
                copy to backup via external software
unlock (via QMP)
fsthaw (optional)

As far as I understand CBT in VMware is not just only a dirty bitmap, but also 
a write generation tracking for blocks (size 64kb or whatever)

I think at the moment I'm worried about getting the basic features out
the door, but I'm not opposed to adding fancier features if there's
justification or demand for them.

Sure, the basic features are most important. I was just thinking of the above scenario to 
interact with a NAS and have Qemu's "help"
to create incremental backups.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]