[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [RFC] Incremental live backup with in memory dirty bitm
From: |
Stefan Hajnoczi |
Subject: |
Re: [Qemu-devel] [RFC] Incremental live backup with in memory dirty bitmap |
Date: |
Mon, 2 Dec 2013 16:48:27 +0100 |
User-agent: |
Mutt/1.5.21 (2010-09-15) |
On Mon, Nov 25, 2013 at 05:59:12PM +0800, Fam Zheng wrote:
> This is an idea about allowing online incremental backup of block
> device, with drive-backup and (proposed here) in-memory block dirty
> bitmap:
>
> 1. We enable a dirty bitmap on a block device, at the start point of
> write tracking:
>
> (QMP) dirty-bitmap-add device=foo name=bitmap0 granularity=4k
>
> Since now, all the writes to "foo" will mark dirty bits on it.
>
> 2.1 Later, when the user wants to export all the dirty data, one
> possible approach is using image fleecing:
>
> # step a. prepare the image fleecing target
> (QMP) blockdev-add backing=foo file.filename=backup.qcow2
> id=backup0 if=none driver=qcow2
>
> # step b. stop the dirty bitmap and start backup job. These two
> steps need to be in a transaction, so the exported data matches the
> exported dirty bitmap
> (QMP) transaction:
> dirty-bitmap-stop device=foo name=bitmap0
> blockdev-backup device=foo sync=none target=backup0
>
> # step c. export the snapshot:
> (QMP) nbd-server-add device=backup0
>
> # step d. export the dirty bitmap as well, so the management
> application can access it and decide what sectors to backup
> (QMP) dirty-bitmap-nbd-add device=foo name=bitmap0
>
> Now the management application can connect to both dirty bitmap
> target and image fleecing target, and read whatever data it is
> interested in.
>
> The tricky part is to export the dirty bitmap through NBD, but it
> may also be useful for persistent the in-memory dirty bitmap too.
>
> 2.2 Alternatively, we can avoid exporting the dirty bitmap through
> NBD, by adding another sync mode "dirty" to drive-backup, and change
> step b to:
>
> # step b. stop the dirty bitmap and start backup job with the
> dirty bitmap
> (QMP) transaction:
> dirty-bitmap-stop device=foo name=bitmap0
> blockdev-backup device=foo sync=dirty bitmap=bitmap0
> target=backup0
>
> With the "dirty" sync mode, drive-backup/blockdev-backup takes a
> dirty bitmap name of the device and writes only dirty data to
> target.
>
> Note that NBD is not necessary for backup, we can use "drive-backup"
> and omit step c as well. Also, the target could be an NBD server, in
> which case we will directly send new data to a remote NBD target.
> This part is fully flexible.
>
> Summary: what we are missing are the dirty bitmap interfaces to
> add/stop/remove/query/export it and it's support of transaction,
> depends on how we want to use it. E.g. for continuity of the
> incremental tracking, the transaction could be:
>
> (QMP) transaction:
> dirty-bitmap-stop device=foo name=bitmap0
> dirty-bitmap-add device=foo name=bitmap1 # Start a new
> bitmap atomically
> blockdev-backup device=foo sync=dirty bitmap=bitmap0
> target=backup0
>
> And the last command is to free the no longer used dirty bitmap:
>
> (QMP) dirty-bitmap-remove device=foo name=bitmap0
>
> So, do these sound good/useful at all? Any comment is welcome!
This sounds like a good idea. Incremental backup has been discussed
periodically and we're in a good position to actually implement it now.
The dirty bitmap object's lifecycle needs to be fully thought through,
including how to make the bitmap persistent. I think a reasonable
approach is flushing the bitmap to disk when QEMU shuts down. The
bitmap file should be marked "in use" by dirty-bitmap-start so the
contents are not trusted after a power failure/crash.
I like the blockdev-backup sync=dirty mode which can be used to stream
dirty blocks to an external NBD server.
The minimal feature set that covers persistent dirty bitmaps is:
1. dirty-bitmap-add/dirty-bitmap-remove
2. blockdev-backup sync=dirty
The user would not be able to query the bitmap itself but they could
receive a copy of changed blocks.
Eric: any thoughts on persistent dirty bitmap APIs?
Stefan
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: [Qemu-devel] [RFC] Incremental live backup with in memory dirty bitmap,
Stefan Hajnoczi <=