[PATCH v4 00/45] Transactional block-graph modifying API

[Vladimir Sementsov-Ogievskiy]

Hi all!

That's a big series, which unites some of my previous ones, and
completes them with necessary additions to finally implement block-graph
modifying API. The series is called "v4" as it inherits
"[PATCH v3 00/11] blockdev-replace" (among other things).

After this series, we have blockdev-add, blockdev-del and
x-blockdev-replace transaction actions, which allows to insert and
remove filters.

Additional challenge is to avoid intermediate permission update. That's
and existing paradigm of block graph modifications: first do all the
modifications and then refresh the permissions. Now we should bring this
paradigm to block-graph modifying transactions: if several graph
modifying commands are sequential in one transaction, permission are
updated after the last of these commands. The application of this is
possibility to correct copy-before-write filter permission requirements
(see last patch).

I now unite all these things into one series because:
 - they depend on each other and I have to rebase them together when
 something needs fix or refactoring
 - just to resend with my new email address
If needed, parts may go in separate, and I can split them again if
necessary.

So, what is here:

1. "[PATCH 00/14] block: cleanup backing and file handling" series,
unchanged:

  block: BlockDriver: add .filtered_child_is_backing field
  block: introduce bdrv_open_file_child() helper
  block/blklogwrites: don't care to remove bs->file child on failure
  test-bdrv-graph-mod: update test_parallel_perm_update test case
  tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing
  test-bdrv-graph-mod: fix filters to be filters
  block: document connection between child roles and
    bs->backing/bs->file
  block/snapshot: stress that we fallback to primary child
  Revert "block: Let replace_child_noperm free children"
  Revert "block: Let replace_child_tran keep indirect pointer"
  Revert "block: Restructure remove_file_or_backing_child()"
  Revert "block: Pass BdrvChild ** to replace_child_noperm"
  block: Manipulate bs->file / bs->backing pointers in .attach/.detach
  block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr

2. implement bdrv_unref_tran() - the key thing to implement blockdev-del
transaction action later.
This part inherits from "[PATCH 00/14] block: blockdev-del force=false".
Still force=false is not realized and qcow2 is untouched, as the target
now is transactional removement.

  block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child
  block: drop bdrv_detach_child()
  block: drop bdrv_remove_filter_or_cow_child
  block: bdrv_refresh_perms(): allow external tran
  block: refactor bdrv_list_refresh_perms to allow any list of nodes
  block: make permission update functions public
  block: add bdrv_try_set_aio_context_tran transaction action
  block: implemet bdrv_unref_tran()

3. Move blockdev.c transactions to util/transactions.c API.

  blockdev: refactor transaction to use Transaction API
  blockdev: transactions: rename some things
  blockdev: qmp_transaction: refactor loop to classic for
  blockdev: transaction: refactor handling transaction properties
  blockdev: qmp_transaction: drop extra generic layer

4. add blockdev-del transaction action

  qapi: block: add blockdev-del transaction action

5. add blockdev-add transaction action
(inherits from "[PATCH 0/2] blockdev-add transaction")

  block: introduce BDRV_O_NOPERM flag
  block: bdrv_insert_node(): use BDRV_O_NOPERM
  qapi: block: add blockdev-add transaction action
  iotests: add blockdev-add-transaction

6. add x-blockdev-replace command and transaction action
(inherits from "[PATCH v3 00/11] blockdev-replace")

  block-backend: blk_root(): drop const specifier on return type
  block/export: add blk_by_export_id()
  block: make bdrv_find_child() function public
  block: bdrv_replace_child_bs(): move to external transaction
  qapi: add x-blockdev-replace command
  qapi: add x-blockdev-replace transaction action
  block: bdrv_get_xdbg_block_graph(): report export ids
  iotests.py: qemu_img_create: use imgfmt by default
  iotests.py: introduce VM.assert_edges_list() method
  iotests.py: add VM.qmp_check() helper
  iotests: add filter-insertion

7. Correct permission scheme of copy-before-write filter, with help of
new design of graph-modifying API.

  block: bdrv_open_inherit: create BlockBackend only when necessary
  block/copy-before-write: correct permission scheme

 block.c                                       | 869 ++++++++++--------
 block/blkdebug.c                              |   9 +-
 block/blklogwrites.c                          |  11 +-
 block/blkreplay.c                             |   7 +-
 block/blkverify.c                             |   9 +-
 block/block-backend.c                         |  10 +-
 block/bochs.c                                 |   7 +-
 block/cloop.c                                 |   7 +-
 block/commit.c                                |   1 +
 block/copy-before-write.c                     |  24 +-
 block/copy-on-read.c                          |   9 +-
 block/crypto.c                                |  11 +-
 block/dmg.c                                   |   7 +-
 block/export/export.c                         |  31 +
 block/filter-compress.c                       |   6 +-
 block/mirror.c                                |   1 +
 block/monitor/block-hmp-cmds.c                |   2 +-
 block/parallels.c                             |   7 +-
 block/preallocate.c                           |   9 +-
 block/qcow.c                                  |   6 +-
 block/qcow2.c                                 |   8 +-
 block/qed.c                                   |   8 +-
 block/raw-format.c                            |   4 +-
 block/replication.c                           |   8 +-
 block/snapshot.c                              |  60 +-
 block/throttle.c                              |   8 +-
 block/vdi.c                                   |   7 +-
 block/vhdx.c                                  |   7 +-
 block/vmdk.c                                  |   7 +-
 block/vpc.c                                   |   7 +-
 blockdev.c                                    | 818 +++++++++--------
 include/block/block.h                         |  72 +-
 include/block/block_int.h                     |  41 +-
 include/block/export.h                        |   1 +
 include/sysemu/block-backend.h                |   3 +-
 qapi/block-core.json                          |  73 +-
 qapi/transaction.json                         |  35 +-
 stubs/blk-by-qdev-id.c                        |   9 +
 stubs/blk-exp-find-by-blk.c                   |   9 +
 stubs/meson.build                             |   2 +
 tests/qemu-iotests/iotests.py                 |  23 +
 .../tests/blockdev-add-transaction            |  52 ++
 .../tests/blockdev-add-transaction.out        |   6 +
 tests/qemu-iotests/tests/filter-insertion     | 253 +++++
 tests/qemu-iotests/tests/filter-insertion.out |   5 +
 tests/qemu-iotests/tests/image-fleecing       |  29 +-
 tests/qemu-iotests/tests/image-fleecing.out   |   2 -
 tests/unit/test-bdrv-drain.c                  |  11 +-
 tests/unit/test-bdrv-graph-mod.c              |  94 +-
 49 files changed, 1715 insertions(+), 990 deletions(-)
 create mode 100644 stubs/blk-by-qdev-id.c
 create mode 100644 stubs/blk-exp-find-by-blk.c
 create mode 100755 tests/qemu-iotests/tests/blockdev-add-transaction
 create mode 100644 tests/qemu-iotests/tests/blockdev-add-transaction.out
 create mode 100755 tests/qemu-iotests/tests/filter-insertion
 create mode 100644 tests/qemu-iotests/tests/filter-insertion.out

-- 
2.35.1