[Qemu-commits] [qemu/qemu] a6f8f9: iotests: Prefer null-co over null-aio

qemu-commits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] a6f8f9: iotests: Prefer null-co over null-aio

From:	Peter Maydell
Subject:	[Qemu-commits] [qemu/qemu] a6f8f9: iotests: Prefer null-co over null-aio
Date:	Mon, 28 Oct 2019 14:12:34 -0700
  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: a6f8f9f82c857cf31cda702e6c62d623e8ddedd5
      
https://github.com/qemu/qemu/commit/a6f8f9f82c857cf31cda702e6c62d623e8ddedd5
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/093
    M tests/qemu-iotests/245

  Log Message:
  -----------
  iotests: Prefer null-co over null-aio

We use null-co basically everywhere in the iotests.  Unless we want to
test null-aio specifically, we should use it instead (for consistency).

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Reviewed-by: Andrey Shinkevich <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6be012252018249d3a2843a0143ee8cca41c276d
      
https://github.com/qemu/qemu/commit/6be012252018249d3a2843a0143ee8cca41c276d
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Allow skipping test cases

case_notrun() does not actually skip the current test case.  It just
adds a "notrun" note and then returns to the caller, who manually has to
skip the test.  Generally, skipping a test case is as simple as
returning from the current function, but not always: For example, this
model does not allow skipping tests already in the setUp() function.

Thus, add a QMPTestCase.case_skip() function that invokes case_notrun()
and then self.skipTest().  To make this work, we need to filter the
information on how many test cases were skipped from the unittest
output.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Andrey Shinkevich <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: e6067a950c44809bbeeb340b2595a5ab844635fd
      
https://github.com/qemu/qemu/commit/e6067a950c44809bbeeb340b2595a5ab844635fd
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Use case_skip() in skip_if_unsupported()

skip_if_unsupported() should use the stronger variant case_skip(),
because this allows it to be used even with setUp() (in a meaningful
way).

In the process, make it explicit what we expect the first argument of
the func_wrapper to be (namely something derived of QMPTestCase).

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Andrey Shinkevich <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 7448be831a6b3a3a7f91d873a52bb2d1ce0bd3d9
      
https://github.com/qemu/qemu/commit/7448be831a6b3a3a7f91d873a52bb2d1ce0bd3d9
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Let skip_if_unsupported accept a function

This lets tests use skip_if_unsupported() with a potentially variable
list of required formats.

Suggested-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Andrey Shinkevich <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: a377dd5170f2327cda746145027540dc823009fb
      
https://github.com/qemu/qemu/commit/a377dd5170f2327cda746145027540dc823009fb
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/093

  Log Message:
  -----------
  iotests: Test driver whitelisting in 093

null-aio may not be whitelisted.  Skip all test cases that require it.

(And skip the whole test if null-co is not whitelisted.)

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Andrey Shinkevich <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 753b31b5f3248be68f2e02b08b4796358b5e7603
      
https://github.com/qemu/qemu/commit/753b31b5f3248be68f2e02b08b4796358b5e7603
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/136

  Log Message:
  -----------
  iotests: Test driver whitelisting in 136

null-aio may not be whitelisted.  Skip all test cases that require it.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Andrey Shinkevich <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 767de537b1268d20dca363a74ba8c8931212d243
      
https://github.com/qemu/qemu/commit/767de537b1268d20dca363a74ba8c8931212d243
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Cache supported_formats()

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Message-id: address@hidden
Reviewed-by: Andrey Shinkevich <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: fed33bd175f663cc8c13f8a490a4f35a19756cfe
      
https://github.com/qemu/qemu/commit/fed33bd175f663cc8c13f8a490a4f35a19756cfe
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M util/hbitmap.c

  Log Message:
  -----------
  hbitmap: handle set/reset with zero length

Passing zero length to these functions leads to unpredicted results.
Zero-length set/reset may occur in active-mirror, on zero-length write
(which is unlikely, but not guaranteed to never happen).

Let's just do nothing on zero-length request.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5c511ac375a859dd12a4c82b56549791570442d3
      
https://github.com/qemu/qemu/commit/5c511ac375a859dd12a4c82b56549791570442d3
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  block/mirror: simplify do_sync_target_write

do_sync_target_write is called from bdrv_mirror_top_do_write after
write/discard operation, all inside active_write/active_write_settle
protecting us from mirror iteration. So the whole area is dirty for
sure, no reason to examine dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: b30168647fea0fc173a0e244d2943fc5575ca511
      
https://github.com/qemu/qemu/commit/b30168647fea0fc173a0e244d2943fc5575ca511
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-backend.c
    M include/sysemu/block-backend.h

  Log Message:
  -----------
  block/block-backend: add blk_co_pwritev_part

Add blk write function with qiov_offset parameter. It's needed for the
following commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: dbdf699cad505be6ba2d4c93066f1dd7aff15aeb
      
https://github.com/qemu/qemu/commit/dbdf699cad505be6ba2d4c93066f1dd7aff15aeb
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  block/mirror: support unaligned write in active mirror

Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up
region in the dirty bitmap, which means that we may not copy some bytes
and assume them copied, which actually leads to producing corrupted
target.

So 9adc1cb49af8d forced dirty bitmap granularity to be
request_alignment for mirror-top filter, so we are not working with
unaligned requests. However forcing large alignment obviously decreases
performance of unaligned requests.

This commit provides another solution for the problem: if unaligned
padding is already dirty, we can safely ignore it, as
1. It's dirty, it will be copied by mirror_iteration anyway
2. It's dirty, so skipping it now we don't increase dirtiness of the
   bitmap and therefore don't damage "synchronicity" of the
   write-blocking mirror.

If unaligned padding is not dirty, we just write it, no reason to touch
dirty bitmap if we succeed (on failure we'll set the whole region
ofcourse, but we loss "synchronicity" on failure anyway).

Note: we need to disable dirty_bitmap, otherwise we will not be able to
see in do_sync_target_write bitmap state before current operation. We
may of course check dirty bitmap before the operation in
bdrv_mirror_top_do_write and remember it, but we don't need active
dirty bitmap for write-blocking mirror anyway.

New code-path is unused until the following commit reverts
9adc1cb49af8d.

Suggested-by: Denis V. Lunev <address@hidden>
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 994b44ab2025c9662f238867e871c8833fd52561
      
https://github.com/qemu/qemu/commit/994b44ab2025c9662f238867e871c8833fd52561
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  Revert "mirror: Only mirror granularity-aligned chunks"

This reverts commit 9adc1cb49af8d4e54f57980b1eed5c0a4b2dafa6.
    "mirror: Only mirror granularity-aligned chunks"

Since previous commit unaligned chunks are supported by
do_sync_target_write.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: c7df3f19d22dc3c13f7ca64d7dc15a4389ee63b8
      
https://github.com/qemu/qemu/commit/c7df3f19d22dc3c13f7ca64d7dc15a4389ee63b8
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/check

  Log Message:
  -----------
  iotests: Introduce $SOCK_DIR

Unix sockets generally have a maximum path length.  Depending on your
$TEST_DIR, it may be exceeded and then all tests that create and use
Unix sockets there may fail.

Circumvent this by adding a new scratch directory specifically for
Unix socket files.  It defaults to a temporary directory (mktemp -d)
that is completely removed after the iotests are done.

(By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which
should be short enough for our use cases.)

Use mkdir -p to create the directory (because it seems right), and do
the same for $TEST_DIR (because there is no reason for that to be
created in any different way).

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 32558ce7a43dc7e8e3d065ac26eb2cdbfa34b36b
      
https://github.com/qemu/qemu/commit/32558ce7a43dc7e8e3d065ac26eb2cdbfa34b36b
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M python/qemu/machine.py
    M python/qemu/qtest.py
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: Store socket files in $SOCK_DIR

iotests.py itself does not store socket files, but machine.py and
qtest.py do.  iotests.py needs to pass the respective path to them, and
they need to adhere to it.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 93b78ea5f698a4f547159fe3700981b2b8c13e78
      
https://github.com/qemu/qemu/commit/93b78ea5f698a4f547159fe3700981b2b8c13e78
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: Add @base_dir to FilePaths etc.

Specifying this optional parameter allows creating temporary files in
other directories than the test_dir; for example in sock_dir.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: dc48bfdf9f38d0a3efeb681161a7636f65dcc652
      
https://github.com/qemu/qemu/commit/dc48bfdf9f38d0a3efeb681161a7636f65dcc652
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/common.filter

  Log Message:
  -----------
  iotests: Filter $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5759322ab0414d277e6f340b2487f37d0f1f3c1e
      
https://github.com/qemu/qemu/commit/5759322ab0414d277e6f340b2487f37d0f1f3c1e
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/241
    M tests/qemu-iotests/common.nbd

  Log Message:
  -----------
  iotests: Let common.nbd create socket in $SOCK_DIR

In addition, drop the nbd_unix_socket assignment in 241 because it does
not really do anything.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: a7552b52331a9eedbc92ba75b47fbacb9f0575f1
      
https://github.com/qemu/qemu/commit/a7552b52331a9eedbc92ba75b47fbacb9f0575f1
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/083
    M tests/qemu-iotests/083.out

  Log Message:
  -----------
  iotests/083: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 46cabce6c2d30c1ff2f3c1a4178f786dcc8ed672
      
https://github.com/qemu/qemu/commit/46cabce6c2d30c1ff2f3c1a4178f786dcc8ed672
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/140
    M tests/qemu-iotests/140.out

  Log Message:
  -----------
  iotests/140: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9a9c7c8f9834f18204a6586e6ddd902dab2b0bf9
      
https://github.com/qemu/qemu/commit/9a9c7c8f9834f18204a6586e6ddd902dab2b0bf9
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/143
    M tests/qemu-iotests/143.out

  Log Message:
  -----------
  iotests/143: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 610dffaa3948c762a1e287410bead51fc876c1a6
      
https://github.com/qemu/qemu/commit/610dffaa3948c762a1e287410bead51fc876c1a6
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/147

  Log Message:
  -----------
  iotests/147: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 14fa704577c9acbdd8c9e117349fededeed5d9c9
      
https://github.com/qemu/qemu/commit/14fa704577c9acbdd8c9e117349fededeed5d9c9
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/181

  Log Message:
  -----------
  iotests/181: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 748f831b2a4e6bcebd99b6fd02612b7d708e61dc
      
https://github.com/qemu/qemu/commit/748f831b2a4e6bcebd99b6fd02612b7d708e61dc
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/182

  Log Message:
  -----------
  iotests/182: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6ab72e5865529654cf357f56e3466247286bc3a7
      
https://github.com/qemu/qemu/commit/6ab72e5865529654cf357f56e3466247286bc3a7
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/183

  Log Message:
  -----------
  iotests/183: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 7310e0bd64026223cc74701bcd20d58937e3578e
      
https://github.com/qemu/qemu/commit/7310e0bd64026223cc74701bcd20d58937e3578e
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/192
    M tests/qemu-iotests/192.out

  Log Message:
  -----------
  iotests/192: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4b4d34f4f79f16f1135705161bb2c2991df9049e
      
https://github.com/qemu/qemu/commit/4b4d34f4f79f16f1135705161bb2c2991df9049e
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/194

  Log Message:
  -----------
  iotests/194: Create sockets in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 2b4af4650c5a668b3a713fa2c54f0abdb7ef54eb
      
https://github.com/qemu/qemu/commit/2b4af4650c5a668b3a713fa2c54f0abdb7ef54eb
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/201

  Log Message:
  -----------
  iotests/201: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 2683ff77dc7f5dac830ba07ab09b0490f49486a6
      
https://github.com/qemu/qemu/commit/2683ff77dc7f5dac830ba07ab09b0490f49486a6
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/205

  Log Message:
  -----------
  iotests/205: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9a1c51e11aaa0be62ac31bd6bca855b1af7d979a
      
https://github.com/qemu/qemu/commit/9a1c51e11aaa0be62ac31bd6bca855b1af7d979a
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/208

  Log Message:
  -----------
  iotests/208: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 73752070be2f07a73a17a398c36223f030ef59ca
      
https://github.com/qemu/qemu/commit/73752070be2f07a73a17a398c36223f030ef59ca
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/209

  Log Message:
  -----------
  iotests/209: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9ea16864f47b55fb5ec787c974c0bd1f659cd498
      
https://github.com/qemu/qemu/commit/9ea16864f47b55fb5ec787c974c0bd1f659cd498
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/222

  Log Message:
  -----------
  iotests/222: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 135a46630683ef79a1c4d4b8f195b65ca65385a5
      
https://github.com/qemu/qemu/commit/135a46630683ef79a1c4d4b8f195b65ca65385a5
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/223

  Log Message:
  -----------
  iotests/223: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5b2da7f7cc5c03e611ca6c70151fbe7275ea3d52
      
https://github.com/qemu/qemu/commit/5b2da7f7cc5c03e611ca6c70151fbe7275ea3d52
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/240

  Log Message:
  -----------
  iotests/240: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: f0e24942fc93708aa971bf4142b97b40d198700c
      
https://github.com/qemu/qemu/commit/f0e24942fc93708aa971bf4142b97b40d198700c
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/267
    M tests/qemu-iotests/267.out

  Log Message:
  -----------
  iotests/267: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: ecc47cb47ccd3da0ad1375d2696997192f953ca3
      
https://github.com/qemu/qemu/commit/ecc47cb47ccd3da0ad1375d2696997192f953ca3
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/common.filter

  Log Message:
  -----------
  iotests: Drop TEST_DIR filter from _filter_nbd

Sockets should be placed into $SOCK_DIR instead of $TEST_DIR, so remove
the $TEST_DIR filter from _filter_nbd.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 3816edd2cb8391a782bde365d7f14705fdb2834d
      
https://github.com/qemu/qemu/commit/3816edd2cb8391a782bde365d7f14705fdb2834d
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-copy.c

  Log Message:
  -----------
  block/block-copy: allocate buffer in block_copy_with_bounce_buffer

Move bounce_buffer allocation block_copy_with_bounce_buffer. This
commit simplifies further work on implementing copying by larger chunks
(of different size) and further asynchronous handling of block_copy
iterations (with help of block/aio_task API).

Allocation works fast, a lot faster than disk io, so it's not a problem
that we now allocate/free bounce_buffer more times. And we anyway will
have to allocate several bounce_buffers for parallel execution of loop
iterations in future.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: b3b7036afbf564268ad7edc10fb28501af0709e9
      
https://github.com/qemu/qemu/commit/b3b7036afbf564268ad7edc10fb28501af0709e9
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-copy.c

  Log Message:
  -----------
  block/block-copy: limit copy_range_size to 16 MiB

Large copy range may imply memory allocation and large io effort, so
using 2G copy range request may be bad idea. Let's limit it to 16 MiB.
It also helps the following patch to refactor copy-with-offload
fallback to copy-with-bounce-buffer.

Note, that total memory usage of backup is still not limited, it will
be fixed in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: e332a726da322c5c68bc34ffb59384e31e36d0ff
      
https://github.com/qemu/qemu/commit/e332a726da322c5c68bc34ffb59384e31e36d0ff
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-copy.c
    M block/trace-events

  Log Message:
  -----------
  block/block-copy: refactor copying

Merge copying code into one function block_copy_do_copy, which only
calls bdrv_ io functions and don't do any synchronization (like dirty
bitmap set/reset).

Refactor block_copy() function so that it takes full decision about
size of chunk to be copied and does all the synchronization (checking
intersecting requests, set/reset dirty bitmaps).

It will help:
 - introduce parallel processing of block_copy iterations: we need to
   calculate chunk size, start async chunk copying and go to the next
   iteration
 - simplify synchronization improvement (like memory limiting in
   further commit and reducing critical section (now we lock the whole
   requested range, when actually we need to lock only dirty region
   which we handle at the moment))

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: f16ba00de946517e5cca4edeeec3e21f8f77132d
      
https://github.com/qemu/qemu/commit/f16ba00de946517e5cca4edeeec3e21f8f77132d
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    A include/qemu/co-shared-resource.h
    M util/Makefile.objs
    A util/qemu-co-shared-resource.c

  Log Message:
  -----------
  util: introduce SharedResource

Introduce an API for some shared splittable resource, like memory.
It's going to be used by backup. Backup uses both read/write io and
copy_range. copy_range may consume memory implictly, so the new API is
abstract: it doesn't allocate any real memory but only hands out
tickets.

The idea is that we have some total amount of something and callers
should wait in coroutine queue if there is not enough of the resource
at the moment.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 7f739d0e5375a934f0eade3cd38e6d33673beec0
      
https://github.com/qemu/qemu/commit/7f739d0e5375a934f0eade3cd38e6d33673beec0
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-copy.c
    M include/block/block-copy.h

  Log Message:
  -----------
  block/block-copy: add memory limit

Currently total allocation for parallel requests to block-copy instance
is unlimited. Let's limit it to 128 MiB.

For now block-copy is used only in backup, so actually we limit total
allocation for backup job.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0e2402452f1f2042923a5206b9ff3e9d70c77811
      
https://github.com/qemu/qemu/commit/0e2402452f1f2042923a5206b9ff3e9d70c77811
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-copy.c
    M include/block/block-copy.h

  Log Message:
  -----------
  block/block-copy: increase buffered copy request

No reason to limit buffered copy to one cluster. Let's allow up to 1
MiB.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: e0dd95e3738c5868ddea932aa0f1998900eeaf58
      
https://github.com/qemu/qemu/commit/e0dd95e3738c5868ddea932aa0f1998900eeaf58
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/nvme.c
    M block/trace-events
    M include/block/nvme.h

  Log Message:
  -----------
  block/nvme: add support for write zeros

Signed-off-by: Maxim Levitsky <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: e87a09d6251b28d1494a3728441d8fdb93a8d57d
      
https://github.com/qemu/qemu/commit/e87a09d6251b28d1494a3728441d8fdb93a8d57d
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/nvme.c
    M block/trace-events

  Log Message:
  -----------
  block/nvme: add support for discard

Signed-off-by: Maxim Levitsky <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: f93c3add3a773e0e3f6277e5517583c4ad3a43c2
      
https://github.com/qemu/qemu/commit/f93c3add3a773e0e3f6277e5517583c4ad3a43c2
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: Do not dereference invalid pointers

mirror_exit_common() may be called twice (if it is called from
mirror_prepare() and fails, it will be called from mirror_abort()
again).

In such a case, many of the pointers in the MirrorBlockJob object will
already be freed.  This can be seen most reliably for s->target, which
is set to NULL (and then dereferenced by blk_bs()).

Cc: address@hidden
Fixes: 737efc1eda23b904fbe0e66b37715fb0e5c3e58b
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5d5b33c08031cfe0e9872bde1f9dcb2215f9b30a
      
https://github.com/qemu/qemu/commit/5d5b33c08031cfe0e9872bde1f9dcb2215f9b30a
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M hw/block/virtio-blk.c
    M hw/net/virtio-net.c
    M include/hw/virtio/virtio.h
    M include/qemu/compiler.h

  Log Message:
  -----------
  include: Move endof() up from hw/virtio/virtio.h

endof() is a useful macro, we can make use of it outside of virtio.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: d8fa8442ad4bdc869afd847ebb8e0fcedff6968c
      
https://github.com/qemu/qemu/commit/d8fa8442ad4bdc869afd847ebb8e0fcedff6968c
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c

  Log Message:
  -----------
  qcow2: Use endof()

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: ecf6c7c0c1e5e6883f0e245a7b47101b69dc8235
      
https://github.com/qemu/qemu/commit/ecf6c7c0c1e5e6883f0e245a7b47101b69dc8235
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Add Error ** to qcow2_read_snapshots()

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: fcf9a6b7288154677f6ee9c5f345c4275170e97d
      
https://github.com/qemu/qemu/commit/fcf9a6b7288154677f6ee9c5f345c4275170e97d
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Keep unknown extra snapshot data

The qcow2 specification says to ignore unknown extra data fields in
snapshot table entries.  Currently, we discard it whenever we update the
image, which is a bit different from "ignore".

This patch makes the qcow2 driver keep all unknown extra data fields
when updating an image's snapshot table.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
[mreitz: Adjusted comments as proposed by Eric]
Signed-off-by: Max Reitz <address@hidden>


  Commit: e0314b56b238ac4f0b74498b7afb016ee85023a6
      
https://github.com/qemu/qemu/commit/e0314b56b238ac4f0b74498b7afb016ee85023a6
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Make qcow2_write_snapshots() public

Updating the snapshot list will be useful when upgrading a v2 image to
v3, so we will need to call this function in qcow2.c.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 722efb0c7c6ae62a9a34d7b256cb9cc8612bda5b
      
https://github.com/qemu/qemu/commit/722efb0c7c6ae62a9a34d7b256cb9cc8612bda5b
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Put qcow2_upgrade() into its own function

This does not make sense right now, but it will make sense once we need
to do more than to just update s->qcow_version.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0a85af351d90e1ae9e06b3bb5290ddf2d5d8b950
      
https://github.com/qemu/qemu/commit/0a85af351d90e1ae9e06b3bb5290ddf2d5d8b950
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Write v3-compliant snapshot list on upgrade

qcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size.  Both are
optional for v2 images, so they may not be present.

qcow2_upgrade() therefore should update the snapshot table to ensure all
entries have these extra data fields.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1727347
Reported-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 8bc584fe035d98b8aca4fcc818617c91df0ed925
      
https://github.com/qemu/qemu/commit/8bc584fe035d98b8aca4fcc818617c91df0ed925
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Separate qcow2_check_read_snapshot_table()

Reading the snapshot table can fail.  That is a problem when we want to
repair the image.

Therefore, stop reading the snapshot table in qcow2_do_open() in check
mode.  Instead, add a new function qcow2_check_read_snapshot_table()
that reads the snapshot table at a later point.  In the future, we want
to handle errors here and fix them.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: fe446b5da225b551fc6493890d437fe4a8ba7552
      
https://github.com/qemu/qemu/commit/fe446b5da225b551fc6493890d437fe4a8ba7552
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Add qcow2_check_fix_snapshot_table()

qcow2_check_read_snapshot_table() can perform consistency checks, but it
cannot fix everything.  Specifically, it cannot allocate new clusters,
because that should wait until the refcount structures are known to be
consistent (i.e., after qcow2_check_refcounts()).  Thus, it cannot call
qcow2_write_snapshots().

Do that in qcow2_check_fix_snapshot_table(), which is called after
qcow2_check_refcounts().

Currently, there is nothing that would set result->corruptions, so this
is a no-op.  A follow-up patch will change that.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: f91f1f159bcaa81a70e860079a73ae7d1220dc0c
      
https://github.com/qemu/qemu/commit/f91f1f159bcaa81a70e860079a73ae7d1220dc0c
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c

  Log Message:
  -----------
  qcow2: Fix broken snapshot table entries

The only case where we currently reject snapshot table entries is when
they have too much extra data.  Fix them with qemu-img check -r all by
counting it as a corruption, reducing their extra_data_size, and then
letting qcow2_check_fix_snapshot_table() do the rest.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 624143355cb6e4149ec27b9b00088aeb958da31d
      
https://github.com/qemu/qemu/commit/624143355cb6e4149ec27b9b00088aeb958da31d
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c

  Log Message:
  -----------
  qcow2: Keep track of the snapshot table length

When repairing the snapshot table, we truncate entries that have too
much extra data.  This frees up space that we do not have to count
towards the snapshot table size.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 099febf3ac37e8d615e90066e515dd9b1d9bba52
      
https://github.com/qemu/qemu/commit/099febf3ac37e8d615e90066e515dd9b1d9bba52
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c

  Log Message:
  -----------
  qcow2: Fix overly long snapshot tables

We currently refuse to open qcow2 images with overly long snapshot
tables.  This patch makes qemu-img check -r all drop all offending
entries past what we deem acceptable.

The user cannot choose which snapshots are removed.  This is fine
because we have chosen the maximum snapshot table size to be so large
(64 MB) that it cannot be reasonably reached.  If the snapshot table
exceeds this size, the image has probably been corrupted in some way; in
this case, it is most important to just make the image usable such that
the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a
plain "check" or "check -r leaks" will not delete any data.)

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: d2b1d1ec734a847ba5d4fba4341a851ec1741d0a
      
https://github.com/qemu/qemu/commit/d2b1d1ec734a847ba5d4fba4341a851ec1741d0a
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c

  Log Message:
  -----------
  qcow2: Repair snapshot table with too many entries

The user cannot choose which snapshots are removed.  This is fine
because we have chosen the maximum snapshot table size to be so large
(65536 entries) that it cannot be reasonably reached.  If the snapshot
table exceeds this size, the image has probably been corrupted in some
way; in this case, it is most important to just make the image usable
such that the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a
plain "check" or "check -r leaks" will not delete any data.)

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: e40e6e88f6c0ac73ba2bf680d4f0e46378c6d0de
      
https://github.com/qemu/qemu/commit/e40e6e88f6c0ac73ba2bf680d4f0e46378c6d0de
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/qcow2-snapshot.c

  Log Message:
  -----------
  qcow2: Fix v3 snapshot table entry compliancy

qcow2 v3 images require every snapshot table entry to have at least 16
bytes of extra data.  If they do not, let qemu-img check -r all fix it.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: fc8ba423ca6b37bee56ec9dc339b44043c39553d
      
https://github.com/qemu/qemu/commit/fc8ba423ca6b37bee56ec9dc339b44043c39553d
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  iotests: Add peek_file* functions

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: f53b25dfd57ef5a65ac82194478b432ba88c9de0
      
https://github.com/qemu/qemu/commit/f53b25dfd57ef5a65ac82194478b432ba88c9de0
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    A tests/qemu-iotests/261
    A tests/qemu-iotests/261.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Test qcow2's snapshot table handling

Add a test how our qcow2 driver handles extra data in snapshot table
entries, and how it repairs overly long snapshot tables.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6b7e8f8b1ce7b41527a7c408491e93f90a442bfc
      
https://github.com/qemu/qemu/commit/6b7e8f8b1ce7b41527a7c408491e93f90a442bfc
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Handle filter truncation like native impl.

Make the filter truncation (passing it through to bs->file) a
first-class citizen and handle it exactly as if it was the filter
driver's native implementation of .bdrv_co_truncate().

I do not see a reason not to, it makes the code a bit shorter, and may
be even more correct because this gets us to finish the write_req that
we prepared before (may be important to e.g. bring dirty bitmaps to the
correct size).

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: bb8160eb78e948b0ef6f5ebbd38ff2a2c3f1dcf3
      
https://github.com/qemu/qemu/commit/bb8160eb78e948b0ef6f5ebbd38ff2a2c3f1dcf3
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/copy-on-read.c

  Log Message:
  -----------
  block/cor: Drop cor_co_truncate()

No other filter driver has a .bdrv_co_truncate() implementation, and
there is no need to because the general block layer code can handle it
just as well.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 26536c7fc25917d1bd13781f81fe3ab039643bff
      
https://github.com/qemu/qemu/commit/26536c7fc25917d1bd13781f81fe3ab039643bff
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/parallels.c
    M block/qcow.c
    M block/qcow2.c

  Log Message:
  -----------
  block: Do not truncate file node when formatting

There is no reason why the format drivers need to truncate the protocol
node when formatting it.  When using the old .bdrv_co_create_ops()
interface, the file will be created with no size option anyway, which
generally gives it a size of 0.  (Exceptions are block devices, which
cannot be truncated anyway.)

When using blockdev-create, the user must have given the file node some
size anyway, so there is no reason why we should override that.

qed is an exception, it needs the file to start completely empty (as
explained by c743849bee7333c7ef256b7e12e34ed6f907064f).

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: c80d8b06cfa59f5d8229379c85dcb2c3bb9c881b
      
https://github.com/qemu/qemu/commit/c80d8b06cfa59f5d8229379c85dcb2c3bb9c881b
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-backend.c
    M block/commit.c
    M block/crypto.c
    M block/file-posix.c
    M block/file-win32.c
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/mirror.c
    M block/nfs.c
    M block/parallels.c
    M block/qcow.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qed.c
    M block/raw-format.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c
    M block/vdi.c
    M block/vhdx-log.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M blockdev.c
    M include/block/block.h
    M include/block/block_int.h
    M include/sysemu/block-backend.h
    M qemu-img.c
    M qemu-io-cmds.c
    M tests/test-block-iothread.c

  Log Message:
  -----------
  block: Add @exact parameter to bdrv_co_truncate()

We have two drivers (iscsi and file-posix) that (in some cases) return
success from their .bdrv_co_truncate() implementation if the block
device is larger than the requested offset, but cannot be shrunk.  Some
callers do not want that behavior, so this patch adds a new parameter
that they can use to turn off that behavior.

This patch just adds the parameter and lets the block/io.c and
block/block-backend.c functions pass it around.  All other callers
always pass false and none of the implementations evaluate it, so that
this patch does not change existing behavior.  Future patches take care
of that.

Suggested-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 82325ae5f2f86ad696db3d66563a078daabc9769
      
https://github.com/qemu/qemu/commit/82325ae5f2f86ad696db3d66563a078daabc9769
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/file-posix.c
    M block/iscsi.c

  Log Message:
  -----------
  block: Evaluate @exact in protocol drivers

We have two protocol drivers that return success when trying to shrink a
block device even though they cannot shrink it.  This behavior is now
only allowed with exact=false, so they should return an error with
exact=true.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: e61a28a9b6b43da6a7a48f6d325fceadf9769388
      
https://github.com/qemu/qemu/commit/e61a28a9b6b43da6a7a48f6d325fceadf9769388
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/crypto.c
    M block/qcow2.c
    M block/raw-format.c

  Log Message:
  -----------
  block: Let format drivers pass @exact

When truncating a format node, the @exact parameter is generally handled
simply by virtue of the format storing the new size in the image
metadata.  Such formats do not need to pass on the parameter to their
file nodes.

There are exceptions, though:
- raw and crypto cannot store the image size, and thus must pass on
  @exact.

- When using qcow2 with an external data file, it just makes sense to
  keep its size in sync with the qcow2 virtual disk (because the
  external data file is the virtual disk).  Therefore, we should pass
  @exact when truncating it.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: e8d04f92378c2de7b464e04469a657fd37eb29ea
      
https://github.com/qemu/qemu/commit/e8d04f92378c2de7b464e04469a657fd37eb29ea
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/parallels.c
    M block/qcow2.c
    M block/qed.c
    M qemu-img.c
    M qemu-io-cmds.c

  Log Message:
  -----------
  block: Pass truncate exact=true where reasonable

This is a change in behavior, so all instances need a good
justification.  The comments added here should explain my reasoning.

qed already had a comment that suggests it always expected
bdrv_truncate()/blk_truncate() to behave as if exact=true were passed
(c743849bee7 came eight months before 55b949c8476), so it was simply
broken until now.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
[mreitz: Changed comment in qed.c to explain why a new QED file must be
         empty, as requested and suggested by Maxim]
Signed-off-by: Max Reitz <address@hidden>


  Commit: 09c5c6de41f27bb128a020d64983adf07b6a256c
      
https://github.com/qemu/qemu/commit/09c5c6de41f27bb128a020d64983adf07b6a256c
  Author: Max Reitz <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  Revert "qemu-img: Check post-truncation size"

This reverts commit 5279b30392da7a3248b320c75f20c61e3a95863c.

We no longer need this check because exact=true forces the block driver
to give the image the exact size requested by the user.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: ba9c45139e2938b8d20ce407db83a31bc9e5066c
      
https://github.com/qemu/qemu/commit/ba9c45139e2938b8d20ce407db83a31bc9e5066c
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M tests/qemu-iotests/264

  Log Message:
  -----------
  qemu-iotests: restrict 264 to qcow2 only

264 is unprepared to run with different formats, for example luks needs
handling keys, cloop doesn't support image creation, vpc creates image
larger than requested (which breaks "Backup completed: 5242880" in test
output).

The test is here to check nbd-reconnect feature and we actually don't
need it for all formats. Let's restrict it to qcow2 only.

Reported-by: Max Reitz <address@hidden>
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: aaffb853359829a37daaf883c773e8320b55c723
      
https://github.com/qemu/qemu/commit/aaffb853359829a37daaf883c773e8320b55c723
  Author: Peter Maydell <address@hidden>
  Date:   2019-10-28 (Mon, 28 Oct 2019)

  Changed paths:
    M block/block-backend.c
    M block/block-copy.c
    M block/commit.c
    M block/copy-on-read.c
    M block/crypto.c
    M block/file-posix.c
    M block/file-win32.c
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/mirror.c
    M block/nfs.c
    M block/nvme.c
    M block/parallels.c
    M block/qcow.c
    M block/qcow2-refcount.c
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h
    M block/qed.c
    M block/raw-format.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c
    M block/trace-events
    M block/vdi.c
    M block/vhdx-log.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M blockdev.c
    M hw/block/virtio-blk.c
    M hw/net/virtio-net.c
    M include/block/block-copy.h
    M include/block/block.h
    M include/block/block_int.h
    M include/block/nvme.h
    M include/hw/virtio/virtio.h
    A include/qemu/co-shared-resource.h
    M include/qemu/compiler.h
    M include/sysemu/block-backend.h
    M python/qemu/machine.py
    M python/qemu/qtest.py
    M qemu-img.c
    M qemu-io-cmds.c
    M tests/qemu-iotests/083
    M tests/qemu-iotests/083.out
    M tests/qemu-iotests/093
    M tests/qemu-iotests/136
    M tests/qemu-iotests/140
    M tests/qemu-iotests/140.out
    M tests/qemu-iotests/143
    M tests/qemu-iotests/143.out
    M tests/qemu-iotests/147
    M tests/qemu-iotests/181
    M tests/qemu-iotests/182
    M tests/qemu-iotests/183
    M tests/qemu-iotests/192
    M tests/qemu-iotests/192.out
    M tests/qemu-iotests/194
    M tests/qemu-iotests/201
    M tests/qemu-iotests/205
    M tests/qemu-iotests/208
    M tests/qemu-iotests/209
    M tests/qemu-iotests/222
    M tests/qemu-iotests/223
    M tests/qemu-iotests/240
    M tests/qemu-iotests/241
    M tests/qemu-iotests/245
    A tests/qemu-iotests/261
    A tests/qemu-iotests/261.out
    M tests/qemu-iotests/264
    M tests/qemu-iotests/267
    M tests/qemu-iotests/267.out
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common.filter
    M tests/qemu-iotests/common.nbd
    M tests/qemu-iotests/common.rc
    M tests/qemu-iotests/group
    M tests/qemu-iotests/iotests.py
    M tests/test-block-iothread.c
    M util/Makefile.objs
    M util/hbitmap.c
    A util/qemu-co-shared-resource.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' 
into staging

Block patches for softfreeze:
- iotest patches
- Improve performance of the mirror block job in write-blocking mode
- Limit memory usage for the backup block job
- Add discard and write-zeroes support to the NVMe host block driver
- Fix a bug in the mirror job
- Prevent the qcow2 driver from creating technically non-compliant qcow2
  v3 images (where there is not enough extra data for snapshot table
  entries)
- Allow callers of bdrv_truncate() (etc.) to determine whether the file
  must be resized to the exact given size or whether it is OK for block
  devices not to shrink

# gpg: Signature made Mon 28 Oct 2019 12:13:53 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "address@hidden"
# gpg: Good signature from "Max Reitz <address@hidden>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-10-28: (69 commits)
  qemu-iotests: restrict 264 to qcow2 only
  Revert "qemu-img: Check post-truncation size"
  block: Pass truncate exact=true where reasonable
  block: Let format drivers pass @exact
  block: Evaluate @exact in protocol drivers
  block: Add @exact parameter to bdrv_co_truncate()
  block: Do not truncate file node when formatting
  block/cor: Drop cor_co_truncate()
  block: Handle filter truncation like native impl.
  iotests: Test qcow2's snapshot table handling
  iotests: Add peek_file* functions
  qcow2: Fix v3 snapshot table entry compliancy
  qcow2: Repair snapshot table with too many entries
  qcow2: Fix overly long snapshot tables
  qcow2: Keep track of the snapshot table length
  qcow2: Fix broken snapshot table entries
  qcow2: Add qcow2_check_fix_snapshot_table()
  qcow2: Separate qcow2_check_read_snapshot_table()
  qcow2: Write v3-compliant snapshot list on upgrade
  qcow2: Put qcow2_upgrade() into its own function
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/9bb73502321d...aaffb8533598
[Prev in Thread]
Current Thread
[Next in Thread]
[Qemu-commits] [qemu/qemu] a6f8f9: iotests: Prefer null-co over null-aio, Peter Maydell <=
Prev by Date: [Qemu-commits] [qemu/qemu] a40dce: virtio: basic structure for packed ring
Next by Date: [Qemu-commits] [qemu/qemu] e6e03d: riscv: Skip checking CSR privilege level in debugg...
Previous by thread: [Qemu-commits] [qemu/qemu] a40dce: virtio: basic structure for packed ring
Next by thread: [Qemu-commits] [qemu/qemu] e6e03d: riscv: Skip checking CSR privilege level in debugg...
Index(es):
- Date
- Thread