[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH v5 00/12] Dirty bitmaps migration

From: Denis V. Lunev
Subject: Re: [Qemu-devel] [PATCH v5 00/12] Dirty bitmaps migration
Date: Thu, 21 May 2015 16:57:05 +0300
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0

On 21/05/15 16:51, Vladimir Sementsov-Ogievskiy wrote:
Hi all.

Hmm. There is an interesting suggestion from Denis Lunev (in CC) about how to drop meta bitmaps and make things easer.


> start migration
disk and memory are migrated, but not dirty bitmaps.
> stop vm
create all necessary bitmaps in destination vm (empty, but with same names and granularities and enabled flag)
> start destination vm
empty bitmaps are tracking now
> start migrating dirty bitmaps. merge them to corresponding bitmaps in destination while bitmaps are migrating, they should be in some kind of 'inconsistent' state. so, we can't start backup or other migration while bitmaps are migrating, but vm is already _running_ on destination.

what do you think about it?

the description is a bit incorrect

- start migration process, perform memory and disk migration
   as usual. VM is still executed at source
- start VM on target. VM on source should be on pause as usual,
   do not finish migration process. Running VM on target "writes"
   normally setting dirty bits as usual
- copy active dirty bitmaps from source to target. This is safe
   as VM on source is not running
- "OR" copied bitmaps with ones running on target
- finish migration process (stop source VM).

Downtime will not be increased due to dirty bitmaps with this
approach, migration process is very simple - plain data copy.


On 13.05.2015 18:29, Vladimir Sementsov-Ogievskiy wrote:
These patches provide dirty bitmap migration feature. Only named dirty
bitmaps are to be migrated. Migration may be enabled using migration

     - rebase on master
     - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class
     - remove rfc, as incremental backup series by John Snow are in

v4 significant changes:
  0001: tiny bugfix: out[i] -> out[i-start], same for 'in'
  0007: set chunk size to 1kb, disable live iteration for
        migrating data < 1mb size.

  tests: only one with md5 sum is here. used function event_wait
         by John Snow. (I hope, you don't mind me just adding this
         function with your 'Signed-off-by')

  rfc: This patch set is based on v13 of
       "block: incremental backup series" by John Snow, which are
       not pushed yet.

  based on v13 of "block: incremental backup series" by John Snow.

  changes from v2:
  removed patch for adding dirty parameter (migration capablities used
    0001: printf's dropped, qapi used
  0002: part0 -> zeroes
  0003: part0 -> zeroes
  0005: dirty_dirty -> meta
        add comments about meta bitmap
          0006: the format is changed, nodes used instead of devices.

  other patches are new.

  rfc: there are two tests. They are the same but using different
  interfaces: md5 checksum of the bitmap last layer in query-block or
  separate query-block-dirty-bitmap with dirty bitmap regions.
  The second form is more appropriate for debugging, the first is more
  appropriate for simple regression control. Which should go to

  1. bug-fixes, that are already in upstream, and renaming of function
  bdrv_reset_dirty_bitmap (which is already in Snow's series) are
  2. bitmap store/restore: the concept renamed to serialization, added
  function hbitmap_deserialize_part0, to not transfer zero blocks
  3. migration dirty parameter: added description comment
  4. Other patches are new.

Actually, in this version of the series I'm trying not use
migration/block.c at all. Instead a separate migration unit is added
in the new file migration/dirty-bitmap.c. Now bitmaps are migrated
like blocks in block migration, they have their "dirty-dirty" bitmaps,
for tracking set/unset changes during migration.

The advantages are:
   - no complications of migration/block.c
   - separate dirty-dirty bitmaps provide handling of "unset's"
   - more effective meta-data/data ratio - no tiny bitmap-blocks.

These patches provide dirty bitmap migration feature. Only named dirty
bitmaps are to be migrated. Migration is made as a part of block
migration in block-migration.c.

Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate
command. If "blk" and "inc" parameters are false when "dirty" is true
block migration is actually skipped: no allocatoions, no bdrv_read's,
no bdrv_write's, only bitmaps are migrated.

The patch set includes two my previous bug fixes, which are necessary
for it. The patch set is based on Incremental backup series by John
Vladimir Sementsov-Ogievskiy (12):
   hbitmap: serialization
   block: BdrvDirtyBitmap serialization interface
   block: tiny refactoring: minimize hbitmap_(set/reset) usage
   block: add meta bitmaps
   block: add bdrv_next_dirty_bitmap()
   qapi: add dirty-bitmaps migration capability
   migration: add migration/block-dirty-bitmap.c
   iotests: maintain several vms in test
   iotests: add add_incoming_migration to VM class
   qapi: add md5 checksum of last dirty bitmap level to query-block
   iotests: add dirty bitmap migration test
   migration/qemu-file: make functions qemu_(get/put)_string public

  block.c                        |  98 +++++-
  include/block/block.h          |  22 ++
  include/migration/block.h      |   1 +
  include/migration/migration.h  |   1 +
  include/migration/qemu-file.h  |  17 +
  include/qemu/hbitmap.h         |  67 ++++
  migration/Makefile.objs        |   2 +-
migration/block-dirty-bitmap.c | 693 +++++++++++++++++++++++++++++++++++++++++
  migration/migration.c          |   9 +
  migration/qemu-file.c          |  18 ++
  qapi-schema.json               |   4 +-
  qapi/block-core.json           |   4 +-
  tests/qemu-iotests/117         |  84 +++++
  tests/qemu-iotests/117.out     |   5 +
  tests/qemu-iotests/group       |   1 +
  tests/qemu-iotests/iotests.py  |  14 +-
  util/hbitmap.c                 | 106 +++++++
  vl.c                           |   1 +
  18 files changed, 1137 insertions(+), 10 deletions(-)
  create mode 100644 migration/block-dirty-bitmap.c
  create mode 100755 tests/qemu-iotests/117
  create mode 100644 tests/qemu-iotests/117.out

reply via email to

[Prev in Thread] Current Thread [Next in Thread]