[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v4 3/4] migration: zero-copy flush only at the end of bitmap
From: |
Peter Xu |
Subject: |
Re: [PATCH v4 3/4] migration: zero-copy flush only at the end of bitmap scanning |
Date: |
Mon, 20 Jun 2022 11:44:19 -0400 |
On Mon, Jun 20, 2022 at 11:23:53AM +0200, Juan Quintela wrote:
> Once discussed this, what I asked in the past is that you are having too
> much dirty memory on zero_copy. When you have a Multiterabyte guest, in
> a single round you have a "potentially" dirty memory on each channel of:
>
> total_amount_memory / number of channels.
>
> In a Multiterabyte guest, this is going to be more that probably in the
> dozens of gigabytes. As far as I know there is no card/driver that will
> benefit for so many pages in zero_copy, and kernel will move to
> synchronous copy at some point. (In older threads, daniel showed how to
> test for this case).
I was wondering whether the kernel needs to cache a lot of messages for
zero copy if we don't flush it for a long time, as recvmsg(MSG_ERRQUEUE)
seems to be fetching one message from the kernel one at a time. And,
whether that queue has a limit in length or something.
Does it mean that when the kernel could have cached enough of these
messages then it'll fallback to the no-zero-copy mode? And probably that's
the way how kernel protects itself from using too much buffer for the error
msgs?
This reminded me - Leo, have you considered adding the patch altogether to
detect the "fallback to non-zero-copy" condition? Because when with it and
when the fallback happens at some point (e.g. when the guest memory is
larger than some value) we'll know.
Thanks,
--
Peter Xu
[PATCH v4 1/4] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability, Leonardo Bras, 2022/06/20
[PATCH v4 2/4] QIOChannelSocket: Fix zero-copy send so socket flush works, Leonardo Bras, 2022/06/20