[PATCH v5 00/10] numa/exec/migration: Fix resizing RAM blocks while migrating

[David Hildenbrand]

v4 has been floating around for a while. Let's see if we can find someone
to merge this; or at least give some more feedback ... all patches have
at least one RB.


I realized that resizing RAM blocks while the guest is being migrated
(precopy: resize while still running on the source, postcopy: resize
 while already running on the target) is buggy. In case of precopy, we
can simply cancel migration. Postcopy handling is more involved. Resizing
can currently happen during a guest reboot, triggered by ACPI rebuilds.

Along with the fixes, some cleanups.

--------------------------------------------------------------------------

Example to highlight one part of the problem:

1. Start a paused VM (where a ramblock resize will trigger when booting):
  sudo build/qemu-system-x86_64 \
       --enable-kvm \
       -S \
       -machine q35,nvdimm=on \
       -smp 1 \
       -cpu host \
       -m size=20G,slots=8,maxmem=22G \
       -object memory-backend-file,id=mem0,mem-path=/tmp/nvdimm,size=256M \
       -device nvdimm,label-size=131072,memdev=mem0,id=nvdimm0,slot=1 \
       -nodefaults \
       -chardev stdio,nosignal,id=serial \
       -device isa-serial,chardev=serial \
       -chardev socket,id=monitor,path=/var/tmp/monitor,server,nowait \
       -mon chardev=monitor,mode=readline \
       -device vmgenid \
       -device intel-iommu \
       -nographic

2. Starting precopy and then starting the VM to trigger resizing during
   precopy:
  QEMU 5.2.95 monitor - type 'help' for more information
  (qemu) migrate -d "exec:gzip -c > STATEFILE.gz"
  QEMU 5.2.95 monitor - type 'help' for more information
  (qemu) cont

3a. Before this series, migration never completes:
  QEMU 5.2.95 monitor - type 'help' for more information
  (qemu) info migrate
  globals:
  store-global-state: on
  only-migratable: off
  send-configuration: on
  send-section-footer: on
  decompress-error-check: on
  clear-bitmap-shift: 18
  Migration status: active
  total time: 43826 ms
  expected downtime: 300 ms
  setup: 5 ms
  transferred ram: 65981 kbytes
  throughput: 8.27 mbps
  remaining ram: 18446744073709551612 kbytes
  total ram: 21234188 kbytes
  duplicate: 5308454 pages
  skipped: 0 pages
  normal: 93 pages
  normal bytes: 372 kbytes
  dirty sync count: 1
  page size: 4 kbytes
  multifd bytes: 0 kbytes
  pages-per-second: 0

4. With this change, migration is properly aborted:
  (qemu) info migrate
  globals:
  store-global-state: on
  only-migratable: off
  send-configuration: on
  send-section-footer: on
  decompress-error-check: on
  clear-bitmap-shift: 18
  Migration status: cancelled
  total time: 0 ms

--------------------------------------------------------------------------

Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Juan Quintela <quintela@redhat.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>

v4 -> v5:
- Rephrased some patch descriptions
- Dropped some patches to reduce the footprint
-- "stubs/ram-block: Remove stubs that are no longer needed"
-- "migration/ram: Tolerate partially changed mappings in postcopy code"
- Removed as already upstream now
-- "migration/ram: Consolidate variable reset after placement in
    ram_load_postcopy()"

v3 -> v4:
- Rebased and retested
- Added RBs

v2 -> v3:
- Rebased on current master
- Added RBs
- "migration/ram: Tolerate partially changed mappings in postcopy code"
-- Extended the comment for the uffdio unregister part.

v1 -> v2:
- "util: vfio-helpers: Factor out and fix processing of existing ram
   blocks"
-- Stringify error
- "migraton/ram: Handle RAM block resizes during precopy"
-- Simplified check if we're migrating on the source
- "exec: Relax range check in ram_block_discard_range()"
-- Added to make discard during resizes actually work
- "migration/ram: Discard new RAM when growing RAM blocks after
   ram_postcopy_incoming_init()"
-- Better checks if in the right postcopy mode.
-- Better patch subject/description/comments
- "migration/ram: Handle RAM block resizes during postcopy"
-- Better comments
-- Adapt to changed postcopy checks
- "migrate/ram: Get rid of "place_source" in ram_load_postcopy()"
-- Dropped, as broken
- "migration/ram: Tolerate partially changed mappings in postcopy code"
-- Better comment / description. Clarify that no implicit wakeup will
   happen
-- Warn on EINVAL (older kernels)
-- Wake up any waiter explicitly

David Hildenbrand (10):
  util: vfio-helpers: Factor out and fix processing of existing ram
    blocks
  numa: Teach ram block notifiers about resizeable ram blocks
  numa: Make all callbacks of ram block notifiers optional
  migration/ram: Handle RAM block resizes during precopy
  exec: Relax range check in ram_block_discard_range()
  migration/ram: Discard RAM when growing RAM blocks after
    ram_postcopy_incoming_init()
  migration/ram: Simplify host page handling in ram_load_postcopy()
  migration/ram: Handle RAM block resizes during postcopy
  migration/multifd: Print used_length of memory block
  migration/ram: Use offset_in_ramblock() in range checks

 hw/core/numa.c             |  41 +++++++++--
 hw/i386/xen/xen-mapcache.c |   7 +-
 include/exec/cpu-common.h  |   1 +
 include/exec/memory.h      |  10 +--
 include/exec/ramblock.h    |  10 +++
 include/exec/ramlist.h     |  13 ++--
 migration/migration.c      |   9 ++-
 migration/migration.h      |   1 +
 migration/multifd.c        |   2 +-
 migration/postcopy-ram.c   |  15 ++++-
 migration/ram.c            | 135 +++++++++++++++++++++++++++++--------
 softmmu/physmem.c          |  26 +++++--
 target/i386/hax/hax-mem.c  |   5 +-
 target/i386/sev.c          |  18 ++---
 util/vfio-helpers.c        |  41 ++++-------
 15 files changed, 241 insertions(+), 93 deletions(-)

-- 
2.30.2