[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-block] [PATCH v11 00/14] block: incremental backup transactions us

From: John Snow
Subject: [Qemu-block] [PATCH v11 00/14] block: incremental backup transactions using BlockJobTxn
Date: Thu, 5 Nov 2015 18:13:06 -0500

Welcome to the Incremental Backup Transactions Newsletter!

What's new?

I replaced the per-action "transactional-cancel" parameter with
a per-transaction paremeter named "completion-mode" which is implemented
as an enum in case we want to add new behaviors in the future, such
as a "jobs only" cancel mode.

For now, it's "grouped" or "individual", and if you use it with actions
that do not support the latent transactional cancel, you will receive
an error for your troubles.

Version 10 primarily changed V7's patches 10-11 and replaced them
with patches 10-12 that are cut a little differently.

This is based on top of the work by Stefan Hajnoczi and Fam Zheng.

Recap: motivation for block job transactions
If an incremental backup block job fails then we reclaim the bitmap so
the job can be retried.  The problem comes when multiple jobs are started as
part of a qmp 'transaction' command.  We need to group these jobs in a
transaction so that either all jobs complete successfully or all bitmaps are

Without transactions, there is a case where some jobs complete successfully and
throw away their bitmaps, making it impossible to retry the backup by rerunning
the command if one of the jobs fails.

How does this implementation work?
These patches add a BlockJobTxn object with the following API:

  txn = block_job_txn_new();
  block_job_txn_add_job(txn, job1);
  block_job_txn_add_job(txn, job2);

The jobs either both complete successfully or they both fail/cancel.  If the
user cancels job1 then job2 will also be cancelled and vice versa.

Jobs objects stay alive waiting for other jobs to complete, even if the
coroutines have returned.  They can be cancelled by the user during this time.
Job blockers are still in effect and no other block job can run on this device
in the meantime (since QEMU currently only allows 1 job per device).  This is
the main drawback to this approach but reasonable since you probably don't want
to run other jobs/operations until you're sure the backup was successful (you
won't be able to retry a failed backup if there's a new job running).


v11: Renamed "err-cancel" to "completion-mode"
     "none" becomes "individual"
     "all" becomes "grouped"

v10: Took series back from Fam. (jsnow)
     Replaced per-action parameter with per-transaction properties.
     Patches 10,11 were split into 10-12.

v9: this version fixes a reference count problem with job->bs,
    in patch 05.

v8: Rebase on to master.
    Minor fixes addressing John Snow's comments.

v7: Add Eric's rev-by in 1, 11.
    Add Max's rev-by in 4, 5, 9, 10, 11.
    Add John's rev-by in 5, 6, 8.
    Fix wording for 6. [John]
    Fix comment of block_job_txn_add_job() in 9. [Max]
    Remove superfluous hunks, and document default value in 11. [Eric]
    Update Makefile dep in 14. [Max]


For convenience, this branch is available at:
https://github.com/jnsnow/qemu.git branch block-transpop

This version is tagged block-transpop-v11:

Fam Zheng (6):
  backup: Extract dirty bitmap handling as a separate function
  blockjob: Introduce reference count and fix reference to job->bs
  blockjob: Add .commit and .abort block job actions
  blockjob: Add "completed" and "ret" in BlockJob
  blockjob: Simplify block_job_finish_sync
  block: Add block job transactions

John Snow (7):
  qapi: Add transaction support to block-dirty-bitmap operations
  iotests: add transactional incremental backup test
  block: rename BlkTransactionState and BdrvActionOps
  block/backup: Rely on commit/abort for cleanup
  block: Add BlockJobTxn support to backup_run
  block: add transactional properties
  iotests: 124 - transactional failure test

Stefan Hajnoczi (1):
  tests: add BlockJobTxn unit test

 block.c                    |  19 +-
 block/backup.c             |  50 ++++--
 block/mirror.c             |   2 +-
 blockdev.c                 | 432 ++++++++++++++++++++++++++++++++++-----------
 blockjob.c                 | 189 ++++++++++++++++----
 docs/bitmaps.md            |   6 +-
 include/block/block.h      |   2 +-
 include/block/block_int.h  |   6 +-
 include/block/blockjob.h   |  85 ++++++++-
 qapi-schema.json           |  56 +++++-
 qemu-img.c                 |   3 -
 qmp-commands.hx            |   2 +-
 tests/Makefile             |   3 +
 tests/qemu-iotests/124     | 182 ++++++++++++++++++-
 tests/qemu-iotests/124.out |   4 +-
 tests/test-blockjob-txn.c  | 250 ++++++++++++++++++++++++++
 16 files changed, 1118 insertions(+), 173 deletions(-)
 create mode 100644 tests/test-blockjob-txn.c


reply via email to

[Prev in Thread] Current Thread [Next in Thread]