[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 637b0a: softmmu: Support concurrent bounce bu
|
From: |
Peter Maydell |
|
Subject: |
[Qemu-commits] [qemu/qemu] 637b0a: softmmu: Support concurrent bounce buffers |
|
Date: |
Tue, 10 Sep 2024 08:21:22 -0700 |
Branch: refs/heads/master
Home: https://github.com/qemu/qemu
Commit: 637b0aa139565cb82a7b9269e62214f87082635c
https://github.com/qemu/qemu/commit/637b0aa139565cb82a7b9269e62214f87082635c
Author: Mattias Nissler <mnissler@rivosinc.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M hw/pci/pci.c
M include/exec/memory.h
M include/hw/pci/pci_device.h
M system/memory.c
M system/physmem.c
Log Message:
-----------
softmmu: Support concurrent bounce buffers
When DMA memory can't be directly accessed, as is the case when
running the device model in a separate process without shareable DMA
file descriptors, bounce buffering is used.
It is not uncommon for device models to request mapping of several DMA
regions at the same time. Examples include:
* net devices, e.g. when transmitting a packet that is split across
several TX descriptors (observed with igb)
* USB host controllers, when handling a packet with multiple data TRBs
(observed with xhci)
Previously, qemu only provided a single bounce buffer per AddressSpace
and would fail DMA map requests while the buffer was already in use. In
turn, this would cause DMA failures that ultimately manifest as hardware
errors from the guest perspective.
This change allocates DMA bounce buffers dynamically instead of
supporting only a single buffer. Thus, multiple DMA mappings work
correctly also when RAM can't be mmap()-ed.
The total bounce buffer allocation size is limited individually for each
AddressSpace. The default limit is 4096 bytes, matching the previous
maximum buffer size. A new x-max-bounce-buffer-size parameter is
provided to configure the limit for PCI devices.
Signed-off-by: Mattias Nissler <mnissler@rivosinc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/r/20240819135455.2957406-1-mnissler@rivosinc.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: b84f06c2bee727b3870b4eeccbe3a45c5aea14c1
https://github.com/qemu/qemu/commit/b84f06c2bee727b3870b4eeccbe3a45c5aea14c1
Author: David Hildenbrand <david@redhat.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M include/exec/ramlist.h
M system/physmem.c
Log Message:
-----------
softmmu/physmem: fix memory leak in dirty_memory_extend()
As reported by Peter, we might be leaking memory when removing the
highest RAMBlock (in the weird ram_addr_t space), and adding a new one.
We will fail to realize that we already allocated bitmaps for more
dirty memory blocks, and effectively discard the pointers to them.
Fix it by getting rid of last_ram_page() and by remembering the number
of dirty memory blocks that have been allocated already.
While at it, let's use "unsigned int" for the number of blocks, which
should be sufficient until we reach ~32 exabytes.
Looks like this leak was introduced as we switched from using a single
bitmap_zero_extend() to allocating multiple bitmaps:
bitmap_zero_extend() relies on g_renew() which should have taken care of
this.
Resolves:
https://lkml.kernel.org/r/CAFEAcA-k7a+VObGAfCFNygQNfCKL=AfX6A4kScq=VSSK0peqPg@mail.gmail.com
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Fixes: 5b82b703b69a ("memory: RCU ram_list.dirty_memory[] for safe RAM hotplug")
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Tested-by: Peter Maydell <peter.maydell@linaro.org>
Cc: qemu-stable@nongnu.org
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: David Hildenbrand <david@redhat.com>
Link: https://lore.kernel.org/r/20240828090743.128647-1-david@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: 58073366da6bc8881b616c5fdad385d1d242e7a7
https://github.com/qemu/qemu/commit/58073366da6bc8881b616c5fdad385d1d242e7a7
Author: Fabiano Rosas <farosas@suse.de>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M .gitlab-ci.d/buildtest.yml
Log Message:
-----------
ci: migration: Don't run python tests in the compat job
The vmstate-checker-script test has a bug that makes it flaky. It was
also committed by mistake and will be removed.
Since the migration-compat job takes the tests from the build-previous
job instead of the current HEAD, neither a fix or a removal of the
test will take effect for this release.
Disable the faulty/undesirable test by taking advantage that it only
runs if the PYTHON environment variable is set. This also disables the
analyze-migration-script test, but this is fine because that test
doesn't have migration compatibility implications.
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20240905185445.8179-1-farosas@suse.de
[peterx: Added a TODO to remove the line after 9.2 release, per thuth]
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: 85da4cbe6e5eb6ba6f31c8b30ee4582625546da7
https://github.com/qemu/qemu/commit/85da4cbe6e5eb6ba6f31c8b30ee4582625546da7
Author: Yuan Liu <yuan1.liu@intel.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M docs/devel/migration/features.rst
A docs/devel/migration/qatzip-compression.rst
Log Message:
-----------
docs/migration: add qatzip compression feature
add Intel QATzip compression method introduction
Reviewed-by: Nanhai Zou <nanhai.zou@intel.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Yuan Liu <yuan1.liu@intel.com>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Link: https://lore.kernel.org/r/20240830232722.58272-2-yichen.wang@bytedance.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: e28ed313c268aeb4e0cefb66dcd215c30e4443fe
https://github.com/qemu/qemu/commit/e28ed313c268aeb4e0cefb66dcd215c30e4443fe
Author: Bryan Zhang <bryan.zhang@bytedance.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M meson.build
M meson_options.txt
M scripts/meson-buildoptions.sh
Log Message:
-----------
meson: Introduce 'qatzip' feature to the build system
Add a 'qatzip' feature, which is automatically disabled, and which
depends on the QATzip library if enabled.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Link: https://lore.kernel.org/r/20240830232722.58272-3-yichen.wang@bytedance.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: 86c6eb1f39cbb7eb0467c114469e98ef699fb515
https://github.com/qemu/qemu/commit/86c6eb1f39cbb7eb0467c114469e98ef699fb515
Author: Bryan Zhang <bryan.zhang@bytedance.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M migration/migration-hmp-cmds.c
M migration/options.c
M migration/options.h
M qapi/migration.json
Log Message:
-----------
migration: Add migration parameters for QATzip
Adds support for migration parameters to control QATzip compression
level.
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Prasad Pandit <pjp@fedoraproject.org>
Link: https://lore.kernel.org/r/20240830232722.58272-4-yichen.wang@bytedance.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: 80484f945989988091c5cd729c3e8bde6c14907a
https://github.com/qemu/qemu/commit/80484f945989988091c5cd729c3e8bde6c14907a
Author: Bryan Zhang <bryan.zhang@bytedance.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M hw/core/qdev-properties-system.c
M migration/meson.build
A migration/multifd-qatzip.c
M migration/multifd.h
M qapi/migration.json
Log Message:
-----------
migration: Introduce 'qatzip' compression method
Adds support for 'qatzip' as an option for the multifd compression
method parameter, and implements using QAT for 'qatzip' compression and
decompression.
Acked-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Prasad Pandit <pjp@fedoraproject.org>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Link: https://lore.kernel.org/r/20240830232722.58272-5-yichen.wang@bytedance.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: afe166d4e8bc33bc448cd573b55d0ac094187d48
https://github.com/qemu/qemu/commit/afe166d4e8bc33bc448cd573b55d0ac094187d48
Author: Bryan Zhang <bryan.zhang@bytedance.com>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M tests/qtest/migration-test.c
Log Message:
-----------
tests/migration: Add integration test for 'qatzip' compression method
Adds an integration test for 'qatzip'.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Link: https://lore.kernel.org/r/20240830232722.58272-6-yichen.wang@bytedance.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: 89bccecdda253c9a1a38921cf9266a4f9655c88c
https://github.com/qemu/qemu/commit/89bccecdda253c9a1a38921cf9266a4f9655c88c
Author: Denis V. Lunev <den@openvz.org>
Date: 2024-09-09 (Mon, 09 Sep 2024)
Changed paths:
M system/cpu-throttle.c
M system/trace-events
Log Message:
-----------
system: improve migration debug
Right now migration_throttle() tracepoint lacks very important
important information, i.e. no one could easily say how much the guest
is throttled. This makes difficult to debug guest quality of service
during migration.
This patch adds one more tracepoint into cpu_throttle_set() which is
actually doing this job.
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Peter Xu <peterx@redhat.com>
CC: Fabiano Rosas <farosas@suse.de>
CC: Paolo Bonzini <pbonzini@redhat.com>
Link: https://lore.kernel.org/r/20240905191941.310592-1-den@openvz.org
Signed-off-by: Peter Xu <peterx@redhat.com>
Commit: a66f28df650166ae8b50c992eea45e7b247f4143
https://github.com/qemu/qemu/commit/a66f28df650166ae8b50c992eea45e7b247f4143
Author: Peter Maydell <peter.maydell@linaro.org>
Date: 2024-09-10 (Tue, 10 Sep 2024)
Changed paths:
M .gitlab-ci.d/buildtest.yml
M docs/devel/migration/features.rst
A docs/devel/migration/qatzip-compression.rst
M hw/core/qdev-properties-system.c
M hw/pci/pci.c
M include/exec/memory.h
M include/exec/ramlist.h
M include/hw/pci/pci_device.h
M meson.build
M meson_options.txt
M migration/meson.build
M migration/migration-hmp-cmds.c
A migration/multifd-qatzip.c
M migration/multifd.h
M migration/options.c
M migration/options.h
M qapi/migration.json
M scripts/meson-buildoptions.sh
M system/cpu-throttle.c
M system/memory.c
M system/physmem.c
M system/trace-events
M tests/qtest/migration-test.c
Log Message:
-----------
Merge tag 'migration-20240909-pull-request' of https://gitlab.com/peterx/qemu
into staging
Migration pull request for 9.2
- Mattias's patch to support concurrent bounce buffers for PCI devices
- David's memory leak fix in dirty_memory_extend()
- Fabiano's CI fix to disable vmstate-static-checker test in compat tests
- Denis's patch that adds one more trace point for cpu throttle changes
- Yichen's multifd qatzip compressor support
# -----BEGIN PGP SIGNATURE-----
#
# iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCZt9VlhIccGV0ZXJ4QHJl
# ZGhhdC5jb20ACgkQO1/MzfOr1wZ+4QD+NPzprFD7RF2DPHT5bdo6NTWFSZxW4dyD
# oFp2vhYDEAYA/A5TTfOh3QpYBOaP2PxztZIZSLgs1bokhp+sLM3/PVsK
# =6JYP
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 09 Sep 2024 21:07:50 BST
# gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706
# gpg: issuer "peterx@redhat.com"
# gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [marginal]
# gpg: aka "Peter Xu <peterx@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706
* tag 'migration-20240909-pull-request' of https://gitlab.com/peterx/qemu:
system: improve migration debug
tests/migration: Add integration test for 'qatzip' compression method
migration: Introduce 'qatzip' compression method
migration: Add migration parameters for QATzip
meson: Introduce 'qatzip' feature to the build system
docs/migration: add qatzip compression feature
ci: migration: Don't run python tests in the compat job
softmmu/physmem: fix memory leak in dirty_memory_extend()
softmmu: Support concurrent bounce buffers
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Compare: https://github.com/qemu/qemu/compare/7bbadc60b58b...a66f28df6501
To unsubscribe from these emails, change your notification settings at
https://github.com/qemu/qemu/settings/notifications