[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 0647d4: qcow2: avoid memcpy(dst, NULL, len)
From: |
GitHub |
Subject: |
[Qemu-commits] [qemu/qemu] 0647d4: qcow2: avoid memcpy(dst, NULL, len) |
Date: |
Tue, 13 Sep 2016 08:00:06 -0700 |
Branch: refs/heads/master
Home: https://github.com/qemu/qemu
Commit: 0647d47cc184da587c76743546b6af6dfdb8f1da
https://github.com/qemu/qemu/commit/0647d47cc184da587c76743546b6af6dfdb8f1da
Author: Stefan Hajnoczi <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/qcow2-cluster.c
M block/qcow2.c
Log Message:
-----------
qcow2: avoid memcpy(dst, NULL, len)
Section "7.1.4 Use of library functions" in the C99 standard says:
If an argument to a function has an invalid value (such as [...]
a null pointer [...]) [...] the behavior is undefined.
Additionally the "searching and sorting" functions are specified as
requiring valid pointer values as described in 7.1.4.
This patch fixes the following sanitizer errors:
block/qcow2.c:1807:41: runtime error: null pointer passed as argument 2,
which is declared to never be null
block/qcow2-cluster.c:86:26: runtime error: null pointer passed as argument
2, which is declared to never be null
Reported-by: Peter Maydell <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 9e909a582916f313d9614ccdf994d4d5ad6e2e0f
https://github.com/qemu/qemu/commit/9e909a582916f313d9614ccdf994d4d5ad6e2e0f
Author: Roman Pen <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/linux-aio.c
Log Message:
-----------
linux-aio: consume events in userspace instead of calling io_getevents
AIO context in userspace is represented as a simple ring buffer, which
can be consumed directly without entering the kernel, which obviously
can bring some performance gain. QEMU does not use timeout value for
waiting for events completions, so we can consume all events from
userspace.
Signed-off-by: Roman Pen <address@hidden>
Message-id: address@hidden
Cc: Stefan Hajnoczi <address@hidden>
Cc: Paolo Bonzini <address@hidden>
Cc: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 3407de572b19a9721d3a107fe5856ab3c40b6f52
https://github.com/qemu/qemu/commit/3407de572b19a9721d3a107fe5856ab3c40b6f52
Author: Roman Pen <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/linux-aio.c
Log Message:
-----------
linux-aio: split processing events function
Prepare processing events function to be called from ioq_submit(),
thus split function on two parts: the first harvests completed IO
requests, the second submits pending requests.
Signed-off-by: Roman Pen <address@hidden>
Message-id: address@hidden
Cc: Stefan Hajnoczi <address@hidden>
Cc: Paolo Bonzini <address@hidden>
Cc: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 0ed93d84edabc7656f5c998ae1a346fe8b94ca54
https://github.com/qemu/qemu/commit/0ed93d84edabc7656f5c998ae1a346fe8b94ca54
Author: Roman Pen <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/linux-aio.c
Log Message:
-----------
linux-aio: process completions from ioq_submit()
In order to reduce completion latency it makes sense to harvest completed
requests ASAP. Very fast backend device can complete requests just after
submission, so it is worth trying to check ring buffer in order to peek
completed requests directly after io_submit() has been called.
Indeed, this patch reduces the completions latencies and increases the
overall throughput, e.g. the following is the percentiles of number of
completed requests at once:
1th 10th 20th 30th 40th 50th 60th 70th 80th 90th 99.99th
Before 2 4 42 112 128 128 128 128 128 128 128
After 1 1 4 14 33 45 47 48 50 51 108
That means, that before the current patch is applied the ring buffer is
observed as full (128 requests were consumed at once) in 60% of calls.
After patch is applied the distribution of number of completed requests
is "smoother" and the queue (requests in-flight) is almost never full.
The fio read results are the following (write results are almost the
same and are not showed here):
Before
------
job: (groupid=0, jobs=8): err= 0: pid=2227: Tue Jul 19 11:29:50 2016
Description : [Emulation of Storage Server Access Pattern]
read : io=54681MB, bw=1822.7MB/s, iops=179779, runt= 30001msec
slat (usec): min=172, max=16883, avg=338.35, stdev=109.66
clat (usec): min=1, max=21977, avg=1051.45, stdev=299.29
lat (usec): min=317, max=22521, avg=1389.83, stdev=300.73
clat percentiles (usec):
| 1.00th=[ 346], 5.00th=[ 596], 10.00th=[ 708], 20.00th=[ 852],
| 30.00th=[ 932], 40.00th=[ 996], 50.00th=[ 1048], 60.00th=[ 1112],
| 70.00th=[ 1176], 80.00th=[ 1256], 90.00th=[ 1384], 95.00th=[ 1496],
| 99.00th=[ 1800], 99.50th=[ 1928], 99.90th=[ 2320], 99.95th=[ 2672],
| 99.99th=[ 4704]
bw (KB /s): min=205229, max=553181, per=12.50%, avg=233278.26,
stdev=18383.51
After
------
job: (groupid=0, jobs=8): err= 0: pid=2220: Tue Jul 19 11:31:51 2016
Description : [Emulation of Storage Server Access Pattern]
read : io=57637MB, bw=1921.2MB/s, iops=189529, runt= 30002msec
slat (usec): min=169, max=20636, avg=329.61, stdev=124.18
clat (usec): min=2, max=19592, avg=988.78, stdev=251.04
lat (usec): min=381, max=21067, avg=1318.42, stdev=243.58
clat percentiles (usec):
| 1.00th=[ 310], 5.00th=[ 580], 10.00th=[ 748], 20.00th=[ 876],
| 30.00th=[ 908], 40.00th=[ 948], 50.00th=[ 1012], 60.00th=[ 1064],
| 70.00th=[ 1080], 80.00th=[ 1128], 90.00th=[ 1224], 95.00th=[ 1288],
| 99.00th=[ 1496], 99.50th=[ 1608], 99.90th=[ 1960], 99.95th=[ 2256],
| 99.99th=[ 5408]
bw (KB /s): min=212149, max=390160, per=12.49%, avg=245746.04,
stdev=11606.75
Throughput increased from 1822MB/s to 1921MB/s, average completion latencies
decreased from 1051us to 988us.
Signed-off-by: Roman Pen <address@hidden>
Message-id: address@hidden
Cc: Stefan Hajnoczi <address@hidden>
Cc: Paolo Bonzini <address@hidden>
Cc: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: b5c7ceaf4ba708cbcb13dc98bf1d351aedea9453
https://github.com/qemu/qemu/commit/b5c7ceaf4ba708cbcb13dc98bf1d351aedea9453
Author: Changlong Xie <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M hw/block/virtio-blk.c
Log Message:
-----------
virtio-blk: rename virtio_device_info to virtio_blk_info
The old one is confusing with @virtio_device_info in virtio.c,
so make it more appropriate.
Signed-off-by: Changlong Xie <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: e9d6456e956898ccdd5d112d2e097e58a9d10b58
https://github.com/qemu/qemu/commit/e9d6456e956898ccdd5d112d2e097e58a9d10b58
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block.c
Log Message:
-----------
block: unblock backup operations in backing file
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Kashyap Chamarthy <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 49d3e828f8cf8500918d4579ab6cd9c0f9584d63
https://github.com/qemu/qemu/commit/49d3e828f8cf8500918d4579ab6cd9c0f9584d63
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/backup.c
A include/block/block_backup.h
Log Message:
-----------
Backup: clear all bitmap when doing block checkpoint
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Signed-off-by: zhanghailiang <address@hidden>
Signed-off-by: Gonglei <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: a8bbee0edf2339e713205ddcb82b75907fbd1049
https://github.com/qemu/qemu/commit/a8bbee0edf2339e713205ddcb82b75907fbd1049
Author: Changlong Xie <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/backup.c
M include/block/block_backup.h
Log Message:
-----------
Backup: export interfaces for extra serialization
Normal backup(sync='none') workflow:
step 1. NBD peformance I/O write from client to server
qcow2_co_writev
bdrv_co_writev
...
bdrv_aligned_pwritev
notifier_with_return_list_notify -> backup_do_cow
bdrv_driver_pwritev // write new contents
step 2. drive-backup sync=none
backup_do_cow
{
wait_for_overlapping_requests
cow_request_begin
for(; start < end; start++) {
bdrv_co_readv_no_serialising //read old contents from Secondary disk
bdrv_co_writev // write old contents to hidden-disk
}
cow_request_end
}
step 3. Then roll back to "step 1" to write new contents to Secondary disk.
And for replication, we must make sure that we only read the old contents from
Secondary disk in order to keep contents consistent.
1) Replication workflow of Secondary
virtio-blk
^
-------> 1 NBD |
|| server 3 replication
|| ^ ^
|| | backing backing |
|| Secondary disk 6<-------- hidden-disk 5 <-------- active-disk 4
|| | ^
|| '-------------------------'
|| drive-backup sync=none 2
Hence, we need these interfaces to implement coarse-grained serialization
between
COW of Secondary disk and the read operation of replication.
Example codes about how to use them:
*#include "block/block_backup.h"
static coroutine_fn int xxx_co_readv()
{
CowRequest req;
BlockJob *job = secondary_disk->bs->job;
if (job) {
backup_wait_for_overlapping_requests(job, start, end);
backup_cow_request_begin(&req, job, start, end);
ret = bdrv_co_readv();
backup_cow_request_end(&req);
goto out;
}
ret = bdrv_co_readv();
out:
return ret;
}
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 258854ad1d3454e0912a1bbaaa4d61790d4e0f79
https://github.com/qemu/qemu/commit/258854ad1d3454e0912a1bbaaa4d61790d4e0f79
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/Makefile.objs
Log Message:
-----------
block: Link backup into block core
Some programs that add a dependency on it will use
the block layer directly.
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Signed-off-by: zhanghailiang <address@hidden>
Signed-off-by: Gonglei <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 68365a3843da8d6ac9e1ad69be0deb8c271d0804
https://github.com/qemu/qemu/commit/68365a3843da8d6ac9e1ad69be0deb8c271d0804
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
A docs/block-replication.txt
Log Message:
-----------
docs: block replication's description
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Signed-off-by: zhanghailiang <address@hidden>
Signed-off-by: Gonglei <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: b49f7ead8d222bcb8df0388f3177002f3e33d046
https://github.com/qemu/qemu/commit/b49f7ead8d222bcb8df0388f3177002f3e33d046
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/mirror.c
M blockdev.c
M include/block/block_int.h
M qemu-img.c
Log Message:
-----------
mirror: auto complete active commit
Auto complete mirror job in background to prevent from
blocking synchronously
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: a6b1d4c081a38c5500fb02be6a4def45e43b0deb
https://github.com/qemu/qemu/commit/a6b1d4c081a38c5500fb02be6a4def45e43b0deb
Author: Changlong Xie <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M configure
Log Message:
-----------
configure: support replication
configure --(enable/disable)-replication to switch replication
support on/off, and it is on by default.
We later introduce replation support.
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 190b9a8b559385b733677224277f9050a0cf76f4
https://github.com/qemu/qemu/commit/190b9a8b559385b733677224277f9050a0cf76f4
Author: Changlong Xie <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M Makefile.objs
M qapi/block-core.json
A replication.c
A replication.h
Log Message:
-----------
replication: Introduce new APIs to do replication operation
This commit introduces six replication interfaces(for block, network etc).
Firstly we can use replication_(new/remove) to create/destroy replication
instances, then in migration we can use replication_(start/stop/do_checkpoint
/get_error)_all to handle all replication operations. More detail please
refer to replication.h
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Signed-off-by: zhanghailiang <address@hidden>
Signed-off-by: Gonglei <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 29ff789060dd12eb11a6af4253957e5b441b6f0a
https://github.com/qemu/qemu/commit/29ff789060dd12eb11a6af4253957e5b441b6f0a
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M block/Makefile.objs
A block/replication.c
Log Message:
-----------
replication: Implement new driver for block replication
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Signed-off-by: zhanghailiang <address@hidden>
Signed-off-by: Gonglei <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: b311046696fa808b9b29eeed090a49a0309adf67
https://github.com/qemu/qemu/commit/b311046696fa808b9b29eeed090a49a0309adf67
Author: Changlong Xie <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M tests/.gitignore
M tests/Makefile.include
A tests/test-replication.c
Log Message:
-----------
tests: add unit test case for replication
[Rename get_error test cases to get_error_all to avoid tripping up
scripts that grep for "error:" in test output. It also reflects the
actual replication API function name better.
-Stefan]
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 82ac554345ff2ca3eb934032f4a8c2baa951eb7b
https://github.com/qemu/qemu/commit/82ac554345ff2ca3eb934032f4a8c2baa951eb7b
Author: Wen Congyang <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M qapi/block-core.json
Log Message:
-----------
support replication driver in blockdev-add
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Signed-off-by: Wang WeiWei <address@hidden>
Signed-off-by: zhanghailiang <address@hidden>
Signed-off-by: Gonglei <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 049105a3c1134bb27f52d3a6ec4e92a3d5278800
https://github.com/qemu/qemu/commit/049105a3c1134bb27f52d3a6ec4e92a3d5278800
Author: Changlong Xie <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M MAINTAINERS
Log Message:
-----------
MAINTAINERS: add maintainer for replication
As per Stefan's suggestion, add Wen and I as co-maintainers
of replication.
Cc: Stefan Hajnoczi <address@hidden>
Signed-off-by: Wen Congyang <address@hidden>
Signed-off-by: Changlong Xie <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: e49f827725d53d2fb1b8ec42db96c442d0caf6cd
https://github.com/qemu/qemu/commit/e49f827725d53d2fb1b8ec42db96c442d0caf6cd
Author: Laurent Vivier <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M tests/libqos/virtio.c
Log Message:
-----------
tests: fix qvirtqueue_kick
vq->avail.idx and vq->avail->ring[] are a 16bit values,
so read and write them with readw()/writew() instead of
readl()/writel().
To read/write a 16bit value with a 32bit accessor works fine
on little-endian CPU but not on big endian CPU.
[An equivalent patch for the writew() calls was also sent by
Zhang Shuai <address@hidden>.
--Stefan]
Signed-off-by: Laurent Vivier <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: dce8921b2baaf95974af8176406881872067adfa
https://github.com/qemu/qemu/commit/dce8921b2baaf95974af8176406881872067adfa
Author: Fam Zheng <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M include/sysemu/iothread.h
M iothread.c
M vl.c
Log Message:
-----------
iothread: Stop threads before main() quits
Right after main_loop ends, we release various things but keep iothread
alive. The latter is not prepared to the sudden change of resources.
Specifically, after bdrv_close_all(), virtio-scsi dataplane get a
surprise at the empty BlockBackend:
(gdb) bt
at /usr/src/debug/qemu-2.6.0/hw/scsi/virtio-scsi.c:543
at /usr/src/debug/qemu-2.6.0/hw/scsi/virtio-scsi.c:577
It is because the d->conf.blk->root is set to NULL, then
blk_get_aio_context() returns qemu_aio_context, whereas s->ctx is still
pointing to the iothread:
hw/scsi/virtio-scsi.c:543:
if (s->dataplane_started) {
assert(blk_get_aio_context(d->conf.blk) == s->ctx);
}
To fix this, let's stop iothreads before doing bdrv_close_all().
Cc: address@hidden
Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>
Commit: 4dfbe3767af503a4cd137b15c8a9d8f30b20a6e9
https://github.com/qemu/qemu/commit/4dfbe3767af503a4cd137b15c8a9d8f30b20a6e9
Author: Peter Maydell <address@hidden>
Date: 2016-09-13 (Tue, 13 Sep 2016)
Changed paths:
M MAINTAINERS
M Makefile.objs
M block.c
M block/Makefile.objs
M block/backup.c
M block/linux-aio.c
M block/mirror.c
M block/qcow2-cluster.c
M block/qcow2.c
A block/replication.c
M blockdev.c
M configure
A docs/block-replication.txt
M hw/block/virtio-blk.c
A include/block/block_backup.h
M include/block/block_int.h
M include/sysemu/iothread.h
M iothread.c
M qapi/block-core.json
M qemu-img.c
A replication.c
A replication.h
M tests/.gitignore
M tests/Makefile.include
M tests/libqos/virtio.c
A tests/test-replication.c
M vl.c
Log Message:
-----------
Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into
staging
Pull request
v2:
* Fixed qcow2 sanitizer warnings [Peter]
* Renamed get_error test cases to get_error_all to avoid tripping "error:"
grep scripts [Peter]
* Added Fam's iothread stop patch
# gpg: Signature made Tue 13 Sep 2016 11:02:30 BST
# gpg: using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <address@hidden>"
# gpg: aka "Stefan Hajnoczi <address@hidden>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* remotes/stefanha/tags/block-pull-request:
iothread: Stop threads before main() quits
tests: fix qvirtqueue_kick
MAINTAINERS: add maintainer for replication
support replication driver in blockdev-add
tests: add unit test case for replication
replication: Implement new driver for block replication
replication: Introduce new APIs to do replication operation
configure: support replication
mirror: auto complete active commit
docs: block replication's description
block: Link backup into block core
Backup: export interfaces for extra serialization
Backup: clear all bitmap when doing block checkpoint
block: unblock backup operations in backing file
virtio-blk: rename virtio_device_info to virtio_blk_info
linux-aio: process completions from ioq_submit()
linux-aio: split processing events function
linux-aio: consume events in userspace instead of calling io_getevents
qcow2: avoid memcpy(dst, NULL, len)
Signed-off-by: Peter Maydell <address@hidden>
Compare: https://github.com/qemu/qemu/compare/fa9701240951...4dfbe3767af5
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Qemu-commits] [qemu/qemu] 0647d4: qcow2: avoid memcpy(dst, NULL, len),
GitHub <=