Re: [PATCH v2 2/2] virtio-mem: Handle preallocation with migration

[David Hildenbrand]

On 25.01.22 14:57, David Hildenbrand wrote:
> During precopy we usually write all plugged ares and essentially
> allocate them. However, there are two corner cases:
> 
> 1) Migrating the zeropage
> 
> When the zeropage gets migrated, we first check if the destination range is
> already zero and avoid performing a write in that case:
> ram_handle_compressed(). If the memory backend, like anonymous RAM or
> most filesystems, populate the shared zeropage when reading a (file) hole,
> we don't preallocate backend memory. In that case, we have to explicitly
> trigger the allocation to allocate actual backend memory.
> 
> 2) Excluding memory ranges during migration
> 
> For example, virtio-balloon free page hinting will exclude some pages
> from getting migrated. In that case, we don't allocate memory for
> plugged ranges when migrating.
> 
> So trigger allocation of all plugged ranges when restoring the device state
> and fail gracefully if allocation fails.
> 
> Handling postcopy is a bit more tricky, as postcopy and preallocation
> are problematic in general. To at least mimic what ordinary
> preallocation does, temporarily try allocating the requested amount
> of memory and fail postcopy in case the requested size on source and
> destination doesn't match. This way, we at least checked that there isn't
> a fundamental configuration issue and that we were able to preallocate
> the required amount of memory at least once, instead of failing
> unrecoverably during postcopy later. However, just as ordinary
> preallocation with postcopy, it's racy.
> 
> Tested-by: Michal Privoznik <mprivozn@redhat.com>
> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Signed-off-by: David Hildenbrand <david@redhat.com>

While this patch improves the situation, I'm going to think about one
prealloc case a bit more:

For precopy with ordinary preallocation, the order on the destination is:

-> Preallocate RAM
-> Restore RAM
-> Restore devices

For precopy with virtio-mem preallocation, the order is:

-> Restore RAM
-> Restore devices
 -> Restore preallocation of virito-mem

The end result is the same if preallocation succeeds. However, if
preallocation would fail, for example, if there are insufficient huge
pages around (i.e., user error), we'll crash with a SIGBUS during RAM
migration, instead of failing earlier with "guest RAM allocation
failed". In both events, the VM can continue running on the source,
however, doing the preallocation earlier would be nicer.

Ideally, we'd migrate the virtio-mem bitmap (+eventually other state)
*before* migrating the RAM state. Saving and restoring it before
saving/restoring RAM. The bitmap is effectively immutable while
migration is active already.


We'd simply preallcoate when restoring the state, which wouldn't require
any postcopy specific handling anymore.

I think it should be doable implementation-wise, I just have to think
about possible implications.

-- 
Thanks,

David / dhildenb