[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2 0/6] migration/ram: Optimize for virtio-mem via RamDiscard

From: David Hildenbrand
Subject: Re: [PATCH v2 0/6] migration/ram: Optimize for virtio-mem via RamDiscardManager
Date: Wed, 28 Jul 2021 19:39:39 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0

On 27.07.21 19:10, Peter Xu wrote:
On Tue, Jul 27, 2021 at 11:25:09AM +0200, David Hildenbrand wrote:
For 2) I see 3 options:

a) Sync everything, fixup the dirty bitmap, never clear the dirty log of
discarded parts. It's fairly simple and straight forward, as I can simply
reuse the existing helper. Something that's discarded will never be dirty,
not even if a misbehaving guest touches memory it shouldn't. [this patch]

b) Sync only populated parts, no need to fixup the dirty bitmap, never clear
the dirty log of discarded parts. It's a bit more complicated but achieves
the same goal as a). [optimization I propose for the future]

c) Sync everything, don't fixup the dirty bitmap, clear the dirty log of
discarded parts initially. There are ways we still might migrate discarded
ranges, for example, if a misbehaving guest touches memory it shouldn't.
[what you propose]

Is my understanding correct? Any reasons why we should chose c) over b) long
term or c) over a) short term?

My major concern is we could do something during sync() for not a very good
reason by looping over virtio-mem bitmap for disgarded ranges - IIUC it should
be destined to be merely no-op if the guest is well-behaved, am I right?

I think yes, as long as we have no initially-set bit stuck somewhere.

Meanwhile, I still have no idea how much overhead the "loop" part could bring.
For a large virtio-mem region with frequent plugged/unplugged mem interacted,
it seems possible to take a while to me..  I have no solid idea yet.

Let's do some math. Assume the worst case on a 1TiB device with a 2MiB block size: We have 524288 blocks == bits. That's precisely a 64k bitmap in virtio-mem. In the worst case, every second bit would be clear ("discarded"). For each clear bit ("discarded"), we would have to clear 512 bits (64 bytes) in the dirty bitmap. That's storing 32 MiB.

So scanning 64 KiB, writing 32 MiB. Certainly not perfect, but I am not sure if it will really matter doing that once on every bitmap sync. I guess the bitmap syncing itself is much more expensive -- and not syncing the discarded ranges (b ) above) would make a bigger impact I guess.

The thing is I still think this extra operation during sync() can be ignored by
simply clear dirty log during bitmap init, then.. why not? :)

I guess clearing the dirty log (especially in KVM) might be more expensive. But, anyhow, we actually want b) long-term :)

Clear dirty bitmap is as simple as "reprotect the pages" functional-wise - if
they are unplugged memory ranges, and they shouldn't be written by the guest
(we still allow reads even for virtio-mem compatibility), then I don't see it
an issue to wr-protect it using clear dirty log when bitmap init.

It still makes sense to me to keep the dirty/clear bitmap in-sync, at least
before your plan b proposal; leaving the dirty bits set forever on unplugged
memory is okay but just sounds a bit weird.

Though my concern is only valid when virtio-mem is used, so I don't have a
strong opinion on it as you maintains virtio-mem. I believe you will always
have a better judgement than me on that. Especially, when/if Dave & Juan have
no problem on that. :)

I'd certainly sleep better at night if I can be 100% sure that a page not to be migrated will not get migrated. :)

I'll play with initial clearing and see how much of a difference it makes code wise. Thanks a lot for your feedback!


David / dhildenb

reply via email to

[Prev in Thread] Current Thread [Next in Thread]