[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-block] ping Re: [PATCH v10 00/12] Dirty bitmaps postcopy migration

From: Vladimir Sementsov-Ogievskiy
Subject: [Qemu-block] ping Re: [PATCH v10 00/12] Dirty bitmaps postcopy migration
Date: Fri, 2 Mar 2018 16:05:46 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0


07.02.2018 18:58, Vladimir Sementsov-Ogievskiy wrote:
Hi all!

There is a new version of dirty bitmap postcopy migration series.

Now it is based on Max's block tree: 
where it needs only one patch: "block: maintain persistent disabled bitmaps",
but I hope it is near to be merged.


clone: tag postcopy-v10 from https://src.openvz.org/scm/~vsementsov/qemu.git

01,02: r-b Fam
03: adjust comments about locking
04: fixed 124 iotest (was broken because of small mistake in 
05: rebased on master, staff from migration_thread is moved to 
migration_iteration_run, so
     drop r-b by John and Juan
06: 2.11->2.12, r-b Fam
07,08,09,: r-b Fam

10: move to device names instead of node names, looks like libvirt don't care 
     same node-names.
     flag AUTOLOAD is ignored for now
     skip automatically inserted nodes, when search for dirty bitmaps
     allow migration of no bitmaps (see in dirty_bitmap_load_header new logic
                                    with nothing variable, which avoids extra 
     handle return code of dirty_bitmap_load_header
     avoid iteration if there are no bitmaps (see new .no_bitmaps field of
     call dirty_bitmap_mig_before_vm_start from process_incoming_migration_bh 
     to enable bitmaps in case of postcopy not actually started.
11: not add r-b Fam
     tiny reorganisation of do_test_migration parameters: remove useless default
     values and make shared_storage to be the last
     disable shared storage test for now, until it will be fixed (it will be 
     series, more related to qcow2 than to migration)
12: r-b Fam

also, "iotests: add default node-name" is dropped, as not more needed.


clone: tag postcopy-v9 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v9

01: r-b John
02: was incomplete, now add here bdrv_reclaim_dirty_bitmap fix
03: new
04: new
05: r-b John
07: fix type in commit message, r-b John
09: add comment about is_active_iterate, r-b Snow and keep Juan's r-b, hope 
comment is ok
10: change copyright to Virtuozzo
     reword comment at the top of the file
     rewrite init_dirty_bitmap_migration, to not do same things twice (John)
       and skip _only_ unnamed bitmaps, error out for unnamed nodes (John)
     use new "locked" state of bitmaps instead of frozen on source vm
     do not support migrating bitmap to existent one with the same name,
       keep only create-new-bitmap way
     break loop in dirty_bitmap_load_complete when bitmap is found
     use bitmap locking instead of context acquire
12: rewrite, to add more cases. (note, that 169 iotest is also in my
     "[PATCH v2 0/3] fix bitmaps migration through shared storage", which 
probably should
     go to qemu-stable. So this patch should rewrite it, but here I make it 
like new patch,
     to simplify review. When "[PATCH v2..." merged I'll rebase this on it), 
drop r-b
13: move to separate test, drop r-b


clone: tag postcopy-v8.1 from https://src.openvz.org/scm/~vsementsov/qemu.git

05: fix compilation, add new version for cmma_save_pending too.


clone: tag postcopy-v8 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v8

- rebased on master
- patches 01-03 from v7 are already merged to master
- patch order is changed to make it possible to merge block/dirty-bitmap patches
   in separate if is needed
01: new patch
03: fixed to use _locked version of bdrv_release_dirty_bitmap
06: qapi-schema.json -> qapi/migration.json
     2.9 -> 2.11
10: protocol changed a bit:
   instead of 1 byte "bitmap enabled flag" this byte becomes just "flags"
   and have "enabled", "persistent" and "autoloading" flags inside.
   also, make all migrated bitmaps to be not persistent (to prevent their
   storing on source vm)
14: new patch

patches status:
01-04 - are only about block/dirty-bitmap and have no r-b. Fam, John, Paolo 
(about bitmap lock),
     please look at. These patches are ok to be merged in separate (but before 
other patches are about migration
05-09 has Juan's r-b (and some of them has John's and Eric's r-bs)
10 - the main patch (dirty bitmaps migration), has no r-b.
11 - preparation for tests, not related to migration directly, has Max's r-b, 
ok to be merged
     separately (but before 12-14)
12-14 - tests, 12 and 13 have Max's r-b, 14 is new


clone: tag postcopy-v7 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v7

- rebased on dirty-bitmap byte-based interfaces
     (based on git://repo.or.cz/qemu/ericb.git branch nbd-byte-dirty-v4)
- migration of persistent bitmaps should fail for shared storage migration for 
   as persistent dirty bitmaps are stored/load on inactivate/invalidate-cache.
   also, even for non-shared storage migration there would be useless saving of 
   bitmaps on source. This all will be optimized later.

01: staff from include/migration/vmstate.h moved to 
include/migration/register.h (rebase)
03: some structural changes due to rebase - drop r-b
04: staff from include/migration/vmstate.h moved to 
include/migration/register.h (rebase)
     staff from include/sysemu/sysemu.h moved to migration/savevm.h (rebase)
05: fix patch header: block -> block/dirty-bitmap
     add locking, drop r-b
06: staff from include/migration/migration.h moved to migration/migration.h 
07: add locking, drop r-b
09: staff from include/migration/qemu-file.h moved to migration/qemu-file.h 
10: staff from include/migration/vmstate.h moved to 
include/migration/register.h (rebase)
11: new patch
12: a lot of changes/fixes (mostly by Fam's comments) + rebase
     header-definition movement
     remove include <assert.h>
     add some includes
     fix/refactor bitmap flags send
     byte-based interface for dirty bitmaps (rebase)
     froze bitmaps on source
     init_dirty_bitmap_migration can return error, if some of bitmaps are 
     bdrv_ref drives with bitmaps
     fprintf -> error_report
     check version_id in _load function


clone: tag postcopy-v6 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v6

rebase on master.

03 - tiny contextual change

12 - little change, but it should be reviewed. Call of 
     (which only initialize mutex) moved from start of 
process_incoming_migration_co (it was
     immediately after "mis = migration_incoming_state_new(f)") to 
     to stay with initialization code.
     I remove r-b's, but hope that this will not be a problem. The only change 
in this patch - is moved
     call of init_dirty_bitmap_incoming_migration.
     I do so because of recent

commit b4b076daf324894dd288cbdb67ff1e3c7434df7b
Author: Juan Quintela <address@hidden>
Date:   Mon Jan 23 22:32:06 2017 +0100

     migration: create Migration Incoming State at init time

15 - add Max's r-b


clone: tag postcopy-v5 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v5

- move 'signed-off' over 'reviewed-by' in patches.

03,04 - add comments. Hope they will be ok for you, so add Juan's r-b.
     If not ok - let me know and I'll resend.

06,08,12 - add Max's r-b
07,09,10,11,12 - add Juan's r-b

14 - used last version of this patch from qcow2-bitmap series with
      Max's r-b. It has contextual changes due to different base.

15 - fix 041 iotest, add default node-name only if path is specified and
      node-name is not specified
16 - handle whitespaces
        s/"exec: cat " + fifo/"exec: cat '" + fifo + "'"/
      fix indentation
      add Max's r-b
17 - fix typos, wrong size in comment, s/md5/sha256/
      add Max's r-b


clone: tag postcopy-v4 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v4

reroll, to fix "failed automatic build test"

rebase on master!

07: since 2.8 -> since 2.9
14: fix build of test-hbitmap
     since 2.8 -> since 2.9 and small rewording of comment (this change was not 
done for
     same patch in may parallels series about qcow2 bitmap extension)


rebased on Max's block branch: https://github.com/XanClic/qemu/commits/block
clone: tag postcopy-v3 from https://src.openvz.org/scm/~vsementsov/qemu.git
online: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=postcopy-v3

01  - r-b by Juan
02  - r-b by Juan and David
04  - fix indent
07  - s/since 2.6/since 2.8/
10  - change variable name s/buf/str/
12  - improve copyright message and move it up
       fix memory loss (thanks to Juan)
       switch from DPRINTF to trace events
14* - switch to sha256 and qcrypto_hash_*
       separate qmp command x-debug-block-dirty-bitmap-sha256
16  - use path_suffix for multi-vm test
       fix indent
       fix copyright
       use x-debug-block-dirty-bitmap-sha256 instead of md5
17  - use x-debug-block-dirty-bitmap-sha256 instead of md5
       remove not existing 170 test from qemu-iotests/group

*Note: patch 14 is also used in my second series 'qcow2 persistent dirty 

some bugs fixed, iotests a bit changed and merged into one test.
based on block-next (https://github.com/XanClic/qemu/commits/block-next)
clone: tag postcopy-v2 from https://src.openvz.org/scm/~vsementsov/qemu.git


These series are derived from my 'Dirty bitmaps migration' series. The
core idea is switch to postcopy migration and drop usage of meta

These patches provide dirty bitmap postcopy migration feature. Only
named dirty bitmaps are to be migrated. Migration may be enabled using
migration capabilities.

The overall method (thanks to John Snow):

1. migrate bitmaps meta data in .save_live_setup
    - create/find related bitmaps on target
    - disable them
    - create successors (anonimous children) only for enabled migrated
2. do nothing in precopy stage
3. just before target vm start: enable successors, created in (1)
4. migrate bitmap data
5. reclaime bitmaps (merge successors to their parents)
6. enable bitmaps (only bitmaps, which was enabled in source)

Some patches are unchnaged from (v7) of 'Dirty bitmaps migration'
(DBMv7). I've left Reviewed-by's for them, if you don't like it, say me
and I'll drop them in the following version.

So, relatively to last DBMv7:

01-04: new patches, splitting common postcopy migration out of ram
        postcopy migration
    05: equal to DBMv7.05
    06: new
    07: equal to DBMv7.06
    08: new
    09: equal to DBMv7.07
    10: new
    11: derived from DBMv7.08, see below
12-15: equal to DBMv7.09-12
    16: derived from DBMv7.13
        - switch from fifo to socket, as postcopy don't work with fifo
          for now
        - change parameters: size, granularity, regions
        - add time.sleep, to wait for postcopy migration phase (bad
          temporary solution.
        - drop Reviewed-by
    17: new

    11: the core patch of the series, it is derived from
        [DBMv7.08: migration: add migration_block-dirty-bitmap.c]
        There are a lot of changes related to switching from precopy to
        postcopy, but functions related to migration stream itself
        (structs, send/load sequences) are mostly unchnaged.

        So, changes, to switch from precopy to postcopy:
        - removed all staff related to meta bitmaps and dirty phase!!!
        - add dirty_bitmap_mig_enable_successors, and call it before
          target vm start in loadvm_postcopy_handle_run
        - add enabled_bitmaps list of bitmaps for

        - enabled flag is send with start bitmap chunk instead of
          completion chunk
        - sectors_per_chunk is calculated directly from CHUNK_SIZE, not
          using meta bitmap granularity

        - dirty_bitmap_save_iterate: remove dirty_phase, move bulk_phase
          to postcopy stage
        - dirty_bitmap_save_pending: remove dirty phase related pending,
          switch pending to non-postcopyable
        - dirty_bitmap_load_start: get enabled flag and prepare
          successors for enabled bitmaps, also add them to
          enabled_bitmaps list
        - dirty_bitmap_load_complete: for enabled bitmaps: merge them
          with successors and enable

        - savevm handlers:
          * remove separate savevm_dirty_bitmap_live_iterate_handlers state
            (it was bad idea, any way), and move its save_live_iterate to
          * add is_active_iterate savevm handler, which allows iterations
            only in postcopy stage (after stopping source vm)
          * add has_postcopy savevm handler. (ofcourse, just returning true)
          * use save_live_complete_postcopy instead of

        Other changes:
        - some debug output changed
        - remove MIN_LIVE_SIZE, is_live_iterative and related staff (it
          was needed to omit iterations if bitmap data is small, possibly
          this should be reimplemented)

Vladimir Sementsov-Ogievskiy (12):
   block/dirty-bitmap: add bdrv_dirty_bitmap_enable_successor()
   block/dirty-bitmap: fix locking in bdrv_reclaim_dirty_bitmap
   block/dirty-bitmap: add _locked version of bdrv_reclaim_dirty_bitmap
   dirty-bitmap: add locked state
   migration: introduce postcopy-only pending
   qapi: add dirty-bitmaps migration capability
   migration: include migrate_dirty_bitmaps in migrate_postcopy
   migration/qemu-file: add qemu_put_counted_string()
   migration: add is_active_iterate handler
   migration: add postcopy migration of dirty bitmaps
   iotests: add dirty bitmap migration test
   iotests: add dirty bitmap postcopy test

  qapi/block-core.json           |   5 +-
  qapi/migration.json            |   6 +-
  include/block/dirty-bitmap.h   |   7 +
  include/migration/misc.h       |   3 +
  include/migration/register.h   |  26 +-
  migration/migration.h          |   4 +
  migration/qemu-file.h          |   2 +
  migration/savevm.h             |   5 +-
  block/dirty-bitmap.c           | 123 +++++--
  blockdev.c                     |  19 ++
  hw/s390x/s390-stattrib.c       |   7 +-
  migration/block-dirty-bitmap.c | 737 +++++++++++++++++++++++++++++++++++++++++
  migration/block.c              |   7 +-
  migration/migration.c          |  32 +-
  migration/qemu-file.c          |  13 +
  migration/ram.c                |   9 +-
  migration/savevm.c             |  20 +-
  vl.c                           |   1 +
  migration/Makefile.objs        |   1 +
  migration/trace-events         |  16 +-
  tests/qemu-iotests/169         | 141 ++++++++
  tests/qemu-iotests/169.out     |   5 +
  tests/qemu-iotests/199         | 105 ++++++
  tests/qemu-iotests/199.out     |   5 +
  tests/qemu-iotests/group       |   2 +
  tests/qemu-iotests/iotests.py  |   7 +-
  26 files changed, 1242 insertions(+), 66 deletions(-)
  create mode 100644 migration/block-dirty-bitmap.c
  create mode 100755 tests/qemu-iotests/169
  create mode 100644 tests/qemu-iotests/169.out
  create mode 100755 tests/qemu-iotests/199
  create mode 100644 tests/qemu-iotests/199.out

Best regards,

reply via email to

[Prev in Thread] Current Thread [Next in Thread]