[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

From: Andrey Gruzdev
Subject: Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots
Date: Thu, 11 Feb 2021 21:28:29 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0

On 11.02.2021 20:32, Peter Xu wrote:
On Thu, Feb 11, 2021 at 07:19:47PM +0300, Andrey Gruzdev wrote:
On 09.02.2021 22:06, David Hildenbrand wrote:

just stumbled over this, quick question:

I recently played with UFFD_WP and notices that write protection is
only effective on pages/ranges that have already pages populated (IOW:
!pte_none() in the kernel).

In case memory was never populated (or was discarded using e.g.,
madvice(DONTNEED)), write-protection will be skipped silently and you
won't get WP events for applicable pages.

So if someone writes to a yet unpoupulated page ("zero"), you won't
get WP events.

I can spot that you do a single uffd_change_protection() on the whole

How are you handling that scenario, or why don't you have to handle
that scenario?

Hi David,

I really wonder if such a problem exists.. If we are talking about a
I immediately ran into this issue with my simplest test cases. :)

write to an unpopulated page, we should get first page fault on
non-present page and populate it with protection bits from
respective vma.
For UFFD_WP vma's  page will be populated non-writable. So we'll get
another page fault on present but read-only page and go to
See the attached test program. Triggers for me on 5.11.0-rc6+ and

gcc -lpthread uffdio_wp.c -o uffdio_wp
WP did not fire

Uncomment the placement of the zeropage just before registering to make
the WP actually trigger. If there is no PTE, there is nothing to

And it makes sense: How should the fault handler know which ranges you
wp-ed, if there is no place to store that information (the PTEs!). The
VMA cannot tell that story, it only knows that someone registered
UFFD_WP to selectively wp some parts.

You might have to register also for MISSING faults and place zero pages.

Looked at the kernel code, agree that we miss WP events for unpopulated
pages, UFFD_WP softbit won't be set in this case. But it doesn't make saved
snapshot inconsistent or introduce security issues. The only side effect is
that we may save updated page instead of zeroed, just increasing snapshot
size. However this guest-physical page has never been touched from the point
of view of saved vCPU/device state and is not a concern.
Oh I just remembered one thing, that Linux should be zeroing pages when
allocating, so even if the page has legacy content it'll be cleared with
__GFP_ZERO allocations.  So yeah it would be harder to have issue at least with
a sensible OS.  I'm not sure about Windows or others, but it could be a common
case.  Then the only overhead is the extra pages we kept in the live snapshot,
which takes some more disk space.

Or there could be firmware running without OS at all, but it should really not
read unallocated pages assuming there must be zero.  It's not a sane behavior
even for a firmware.

Often (at least on desktop Windows guests) only a small part of RAM has ever
been allocated by guest. Migration code needs to read each guest-physical
page, so we'll have a lot of additional UFFD events, much more MISSING
events then WP-faults.

And the main problem is that adding MISSING handler is impossible in current
single-threaded snapshot code. We'll get an immediate deadlock on iterative
page read.
Right.  We'll need to rework the design but just for saving a bunch of snapshot
image disk size.  So now I agree with you, let's keep this in mind, but maybe
it isn't worth a fix for now, at least until we figure something really broken.

Andrey, do you think we should still mention this issue into the todo list of
the wiki page of live snapshot?


Yes, even if the page happens to be overwritten, it's overwritten by the same VM so
no security boundaries are crossed. And no machine code can assume that RAM content
is zeroed on power-on or reset so our snapshot state stays quite consistent.

Agree we should keep it in mind, but IMHO adding MISSING handler and running separate
thread would make performance worse.. So I doubt it's worth adding this to TODO list..

Andrey Gruzdev, Principal Engineer
Virtuozzo GmbH  +7-903-247-6397

reply via email to

[Prev in Thread] Current Thread [Next in Thread]