[Qemu-devel] [RFC PATCH v2 00/23] qcow2: Delayed COW

[Kevin Wolf]

During the last few releases we have got rid of most of the overhead of
metadata writes during cluster allocation. What's left is the COW for
unaligned allocating write requests, and it's quite expensive.

In the general case, this cost cannot be avoided. However, if we're
lucky enough that before the next flush the data copied during COW
would be overwritten, we can do without the COW. Sequential writes
always overwrite the COW area at the end of the cluster immediately,
so delaying the COW a bit and cancelling it if it's overwritten is a
worthwhile optimisation.


iozone results with and without this series show significant difference
for allocating writes:

                                                         random  random
           KB  reclen   write rewrite    read    reread    read   write

base    65536       8    1727    1945    12546    12539    2449    1836
patch   65536       8    1934    1949    12263    12521    2463    1796

base  1048576     256   22344   38437   105823   106135   37743   32167
patch 1048576     256   35989   38542   105231   105994   38301   33036


Changes in v2:
- Many of the v1 patches that make sense independently have already gone
  into qemu.git
- Half of the series that basically rewrites qcow2_alloc_cluster_offset()
  is new. This was required in order to make the really interesting patch
  in this series ("qcow2: Cancel COW when overwritten") correct.
- Addressed a few comments on v1 by Paolo and Blue

Still to do:
- Figure out what to do with failing qemu-iotests cases that rely on a
  specific order in which requests complete. This series can very
  obviously change that behaviour. Maybe we should redirect the output
  of all qemu-io instances to /dev/null if they use AIO and the output
  is non-deterministic therefore.

Kevin Wolf (23):
  qcow2: Handle dependencies earlier
  qcow2: Improve check for overlapping allocations
  qcow2: Change handle_dependency to byte granularity
  qcow2: Decouple cluster allocation from cluster reuse code
  qcow2: Factor out handle_alloc()
  qcow2: handle_alloc(): Get rid of nb_clusters parameter
  qcow2: handle_alloc(): Get rid of keep_clusters parameter
  qcow2: Finalise interface of handle_alloc()
  qcow2: Clean up handle_alloc()
  qcow2: Factor out handle_copied()
  qcow2: handle_copied(): Get rid of nb_clusters parameter
  qcow2: handle_copied(): Get rid of keep_clusters parameter
  qcow2: handle_copied(): Implement non-zero host_offset
  qcow2: Use byte granularity in qcow2_alloc_cluster_offset()
  qcow2: Allow requests with multiple l2metas
  qcow2: Reading from areas not in L2 tables yet
  qcow2: Move COW and L2 update into own coroutine
  qcow2: Delay the COW
  qcow2: Add error handling to the l2meta coroutine
  qcow2: Cancel COW when overwritten
  qemu-iotests: Another concurrent multicluster allocation case
  qcow2: Move cluster gathering to a non-looping loop
  qcow2: Gather clusters in a looping loop

 block.c                    |    5 +
 block/qcow2-cluster.c      |  707 ++++++++++++++++++++++++++++++++++----------
 block/qcow2.c              |  209 ++++++++++++-
 block/qcow2.h              |  110 +++++++
 include/block/block_int.h  |    3 +
 tests/qemu-iotests/038.out |   10 +-
 tests/qemu-iotests/046     |   19 ++
 tests/qemu-iotests/046.out |   16 +
 trace-events               |    4 +
 9 files changed, 905 insertions(+), 178 deletions(-)

-- 
1.7.6.5