[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

From: David Hildenbrand
Subject: Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots
Date: Thu, 11 Feb 2021 20:21:34 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0

On 09.02.21 19:38, Andrey Gruzdev wrote:
On 09.02.2021 15:37, David Hildenbrand wrote:
On 21.01.21 16:24, andrey.gruzdev--- via wrote:
This patch series is a kind of 'rethinking' of Denis Plotnikov's
ideas he's
implemented in his series '[PATCH v0 0/4] migration: add background

Currently the only way to make (external) live VM snapshot is using
dirty page logging migration mechanism. The main problem is that it
tends to
produce a lot of page duplicates while running VM goes on updating
saved pages. That leads to the fact that vmstate image size is
commonly several
times bigger then non-zero part of virtual machine's RSS. Time
required to
converge RAM migration and the size of snapshot image severely depend
on the
guest memory write rate, sometimes resulting in unacceptably long
creation time and huge image size.

This series propose a way to solve the aforementioned problems. This
is done
by using different RAM migration mechanism based on UFFD write
management introduced in v5.7 kernel. The migration strategy is to
guest RAM content using write-protection and iteratively release
for memory ranges that have already been saved to the migration stream.
At the same time we read in pending UFFD write fault events and save
pages out-of-order with higher priority.


just stumbled over this, quick question:

I recently played with UFFD_WP and notices that write protection is
only effective on pages/ranges that have already pages populated (IOW:
!pte_none() in the kernel).

In case memory was never populated (or was discarded using e.g.,
madvice(DONTNEED)), write-protection will be skipped silently and you
won't get WP events for applicable pages.

So if someone writes to a yet unpoupulated page ("zero"), you won't
get WP events.

I can spot that you do a single uffd_change_protection() on the whole

How are you handling that scenario, or why don't you have to handle
that scenario?

Hi David,

I really wonder if such a problem exists.. If we are talking about a
write to an unpopulated page, we should get first page fault on
non-present page and populate it with protection bits from respective vma.
For UFFD_WP vma's  page will be populated non-writable. So we'll get
another page fault on present but read-only page and go to handle_userfault.


here is another fun issue.

Assume you

1. Have a populated page, with some valuable content
2. WP protected the page
3. madvise(DONTNEED) that page
4. Write to the page

On write access, you won't get a WP event!

Instead, you will get a UFFD_EVENT_REMOVE during 3. But you cannot stop that event (dont wake), so you cannot simply defer as you can do with WP events.

So if the guest inflates the balloon (including balloon page migration in Linux) or free-page-reporting reports a free page while snapshotting is active, you won't be able to save the old content before it is zapped and your snapshot misses pages with actual content.

Something similar would happen with virtio-mem when unplugging blocks, however, it does not discard any pages while migration is active.

Snapshotting seems to be incompatible with concurrent discards via virtio-balloon. You might want to inhibit ballooning while snapshotting is active in

hw/virtio/virtio-balloon.c:virtio_balloon_inhibited() just as we do for postcopy.


David / dhildenb

reply via email to

[Prev in Thread] Current Thread [Next in Thread]