[Qemu-commits] [qemu/qemu] 9484ad: copy-before-write: allow specifying minimum cluste...

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 9484ad6c17a735284d4ff75ed3140ded9e0065fd
      
https://github.com/qemu/qemu/commit/9484ad6c17a735284d4ff75ed3140ded9e0065fd
  Author: Fiona Ebner <f.ebner@proxmox.com>
  Date:   2024-09-30 (Mon, 30 Sep 2024)

  Changed paths:
    M block/block-copy.c
    M block/copy-before-write.c
    M include/block/block-copy.h
    M qapi/block-core.json

  Log Message:
  -----------
  copy-before-write: allow specifying minimum cluster size

In the context of backup fleecing, discarding the source will not work
when the fleecing image has a larger granularity than the one used for
block-copy operations (can happen if the backup target has smaller
cluster size), because cbw_co_pdiscard_snapshot() will align down the
discard requests and thus effectively ignore then.

To make @discard-source work in such a scenario, allow specifying the
minimum cluster size used for block-copy operations and thus in
particular also the granularity for discard requests to the source.

The type 'size' (corresponding to uint64_t in C) is used in QAPI to
rule out negative inputs and for consistency with already existing
@cluster-size parameters. Since block_copy_calculate_cluster_size()
uses int64_t for its result, a check that the input is not too large
is added in block_copy_state_new() before calling it. The calculation
in block_copy_calculate_cluster_size() is done in the target int64_t
type.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: Markus Armbruster <armbru@redhat.com> (QAPI schema)
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-Id: <20240711120915.310243-2-f.ebner@proxmox.com>
[vsementsov: switch version to 9.2 in QAPI doc]
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: 6252deb244b2d60d03545c17eec548a43eb3aefa
      
https://github.com/qemu/qemu/commit/6252deb244b2d60d03545c17eec548a43eb3aefa
  Author: Fiona Ebner <f.ebner@proxmox.com>
  Date:   2024-09-30 (Mon, 30 Sep 2024)

  Changed paths:
    M block/backup.c
    M block/copy-before-write.c
    M block/copy-before-write.h
    M blockdev.c
    M qapi/block-core.json

  Log Message:
  -----------
  backup: add minimum cluster size to performance options

In the context of backup fleecing, discarding the source will not work
when the fleecing image has a larger granularity than the one used for
block-copy operations (can happen if the backup target has smaller
cluster size), because cbw_co_pdiscard_snapshot() will align down the
discard requests and thus effectively ignore then.

To make @discard-source work in such a scenario, allow specifying the
minimum cluster size used for block-copy operations and thus in
particular also the granularity for discard requests to the source.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: Markus Armbruster <armbru@redhat.com> (QAPI schema)
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-Id: <20240711120915.310243-3-f.ebner@proxmox.com>
[vsementsov: switch version to 9.2 in QAPI doc]
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: 6475155d519209c80fdda53e05130365aa769838
      
https://github.com/qemu/qemu/commit/6475155d519209c80fdda53e05130365aa769838
  Author: Fiona Ebner <f.ebner@proxmox.com>
  Date:   2024-09-30 (Mon, 30 Sep 2024)

  Changed paths:
    M block/copy-before-write.c
    M block/reqlist.c

  Log Message:
  -----------
  block/reqlist: allow adding overlapping requests

Allow overlapping request by removing the assert that made it
impossible. There are only two callers:

1. block_copy_task_create()

It already asserts the very same condition before calling
reqlist_init_req().

2. cbw_snapshot_read_lock()

There is no need to have read requests be non-overlapping in
copy-before-write when used for snapshot-access. In fact, there was no
protection against two callers of cbw_snapshot_read_lock() calling
reqlist_init_req() with overlapping ranges and this could lead to an
assertion failure [1].

In particular, with the reproducer script below [0], two
cbw_co_snapshot_block_status() callers could race, with the second
calling reqlist_init_req() before the first one finishes and removes
its conflicting request.

[0]:

> #!/bin/bash -e
> dd if=/dev/urandom of=/tmp/disk.raw bs=1M count=1024
> ./qemu-img create /tmp/fleecing.raw -f raw 1G
> (
> ./qemu-system-x86_64 --qmp stdio \
> --blockdev raw,node-name=node0,file.driver=file,file.filename=/tmp/disk.raw \
> --blockdev 
> raw,node-name=node1,file.driver=file,file.filename=/tmp/fleecing.raw \
> <<EOF
> {"execute": "qmp_capabilities"}
> {"execute": "blockdev-add", "arguments": { "driver": "copy-before-write", 
> "file": "node0", "target": "node1", "node-name": "node3" } }
> {"execute": "blockdev-add", "arguments": { "driver": "snapshot-access", 
> "file": "node3", "node-name": "snap0" } }
> {"execute": "nbd-server-start", "arguments": {"addr": { "type": "unix", 
> "data": { "path": "/tmp/nbd.socket" } } } }
> {"execute": "block-export-add", "arguments": {"id": "exp0", "node-name": 
> "snap0", "type": "nbd", "name": "exp0"}}
> EOF
> ) &
> sleep 5
> while true; do
> ./qemu-nbd -d /dev/nbd0
> ./qemu-nbd -c /dev/nbd0 nbd:unix:/tmp/nbd.socket:exportname=exp0 -f raw -r
> nbdinfo --map 'nbd+unix:///exp0?socket=/tmp/nbd.socket'
> done

[1]:

> #5  0x000071e5f0088eb2 in __GI___assert_fail (...) at ./assert/assert.c:101
> #6  0x0000615285438017 in reqlist_init_req (...) at ../block/reqlist.c:23
> #7  0x00006152853e2d98 in cbw_snapshot_read_lock (...) at 
> ../block/copy-before-write.c:237
> #8  0x00006152853e3068 in cbw_co_snapshot_block_status (...) at 
> ../block/copy-before-write.c:304
> #9  0x00006152853f4d22 in bdrv_co_snapshot_block_status (...) at 
> ../block/io.c:3726
> #10 0x000061528543a63e in snapshot_access_co_block_status (...) at 
> ../block/snapshot-access.c:48
> #11 0x00006152853f1a0a in bdrv_co_do_block_status (...) at ../block/io.c:2474
> #12 0x00006152853f2016 in bdrv_co_common_block_status_above (...) at 
> ../block/io.c:2652
> #13 0x00006152853f22cf in bdrv_co_block_status_above (...) at 
> ../block/io.c:2732
> #14 0x00006152853d9a86 in blk_co_block_status_above (...) at 
> ../block/block-backend.c:1473
> #15 0x000061528538da6c in blockstatus_to_extents (...) at ../nbd/server.c:2374
> #16 0x000061528538deb1 in nbd_co_send_block_status (...) at 
> ../nbd/server.c:2481
> #17 0x000061528538f424 in nbd_handle_request (...) at ../nbd/server.c:2978
> #18 0x000061528538f906 in nbd_trip (...) at ../nbd/server.c:3121
> #19 0x00006152855a7caf in coroutine_trampoline (...) at 
> ../util/coroutine-ucontext.c:175

Cc: qemu-stable@nongnu.org
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-Id: <20240712140716.517911-1-f.ebner@proxmox.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: e84af3eb727d2c6de00693c48fbec1d6e68fbf9a
      
https://github.com/qemu/qemu/commit/e84af3eb727d2c6de00693c48fbec1d6e68fbf9a
  Author: Dr. David Alan Gilbert <dave@treblig.org>
  Date:   2024-09-30 (Mon, 30 Sep 2024)

  Changed paths:
    M block/aio_task.c
    M include/block/aio_task.h

  Log Message:
  -----------
  block: Remove unused aio_task_pool_empty

aio_task_pool_empty has been unused since it was added in
  6e9b225f73 ("block: introduce aio task pool")

Remove it.

Signed-off-by: Dr. David Alan Gilbert <dave@treblig.org>
Message-Id: <20240917002007.330689-1-dave@treblig.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: b74987cd3bf9bd4bf452ed371d864cbd1dccb08e
      
https://github.com/qemu/qemu/commit/b74987cd3bf9bd4bf452ed371d864cbd1dccb08e
  Author: Dr. David Alan Gilbert <dave@treblig.org>
  Date:   2024-09-30 (Mon, 30 Sep 2024)

  Changed paths:
    M include/qemu/co-shared-resource.h
    M util/qemu-co-shared-resource.c

  Log Message:
  -----------
  util/co-shared-resource: Remove unused co_try_get_from_shres

co_try_get_from_shres hasn't been used since it was added in
  55fa54a789 ("co-shared-resource: protect with a mutex")

(Everyone uses the _locked version)
Remove it.

Signed-off-by: Dr. David Alan Gilbert <dave@treblig.org>
Message-Id: <20240918124220.27871-1-dave@treblig.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: 72cb4806fdb0c5d7efca181dedfd46bed2acc9e7
      
https://github.com/qemu/qemu/commit/72cb4806fdb0c5d7efca181dedfd46bed2acc9e7
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2024-09-30 (Mon, 30 Sep 2024)

  Changed paths:
    M block/aio_task.c
    M block/backup.c
    M block/block-copy.c
    M block/copy-before-write.c
    M block/copy-before-write.h
    M block/reqlist.c
    M blockdev.c
    M include/block/aio_task.h
    M include/block/block-copy.h
    M include/qemu/co-shared-resource.h
    M qapi/block-core.json
    M util/qemu-co-shared-resource.c

  Log Message:
  -----------
  Merge tag 'pull-block-jobs-2024-09-30' of https://gitlab.com/vsementsov/qemu 
into staging

Block-jobs: improve backup fleecing

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCgAdFiEEi5wmzbL9FHyIDoahVh8kwfGfefsFAmb6WsYACgkQVh8kwfGf
# eftECRAAgRFGeVSE8+fgibsHH4TixG41Rg/4gi09KZuhNHCDQXiVWlpNf4isnjcg
# K3PAZh5XSr+ClYFzbwqrjnbjGOZDJRpfb5KAnWK2GhhbBrwFBAussJ/njOg3IB3V
# 1Qtm5cc6aKrFW4dybqWW96gZZkIxFuSOW0jAjuVh4r0ZfK03Xytm5WITYg7CLnnQ
# WqnseoQuj9HvT3F2A3HnRk4tIlgEb2eWppDhX8DiW/IO4C1ScfYpyP2cvUROCCGF
# GIJKv7/GRYosSUx4nsCCx1y7ISX0A5m1sCBlWbnQmYLIaJ/JpUYA2Tqxyfy3LTRE
# IJK/NcWcR9fCy2W1hNQJaKfIKoA7EdfqSvCFQmBHrEmtRYslejNRB3PjIUzth7Oh
# knfHxKSiXyGBdikaE5a//MUBK74pc9eC9zuIvIzxdLYxBTTGKZlTuhxG28QBoLO3
# /IcPd22loEZwh6PRgcFw7olsHo9lhfbiRua8dKwj384yJKFGU4s4QghSqRLxBHjP
# xnqQrU8uE3UOIVMY9ahsMxwUzzYGbwi0vQw98qLm26vcQBZRD1C9L97M9yCCrhQ4
# jfWyMsqdODA6V3IRpIV+JBL5KvRkC1fuoAw6Oh5R2r3K/WpX74gthATna9+/wGno
# 0wPiaEvBox8y6zLKAZe4MEA6phX0tb5In6OQB2JlILHa7gds4x4=
# =LqMR
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 30 Sep 2024 09:01:10 BST
# gpg:                using RSA key 8B9C26CDB2FD147C880E86A1561F24C1F19F79FB
# gpg: Good signature from "Vladimir Sementsov-Ogievskiy 
<vsementsov@yandex-team.ru>" [unknown]
# gpg:                 aka "Vladimir Sementsov-Ogievskiy 
<vsementsov@virtuozzo.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8B9C 26CD B2FD 147C 880E  86A1 561F 24C1 F19F 79FB

* tag 'pull-block-jobs-2024-09-30' of https://gitlab.com/vsementsov/qemu:
  util/co-shared-resource: Remove unused co_try_get_from_shres
  block: Remove unused aio_task_pool_empty
  block/reqlist: allow adding overlapping requests
  backup: add minimum cluster size to performance options
  copy-before-write: allow specifying minimum cluster size

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/3b14a767eaca...72cb4806fdb0

To unsubscribe from these emails, change your notification settings at 
https://github.com/qemu/qemu/settings/notifications