On Thu, Apr 11, 2013 at 11:18:56AM -0400, Michael R. Hines wrote:
First of all,
I know it's a hard habit to break but could you
please stop stop top-posting?
this whole argument should not even exist for the
following reason:
Page registrations are supposed to be *rare* - once a page is registered, it
is registered for life. There is nothing in the design that says a page must
be "unregistered" and I do not believe anybody is proposing that.
Hmm proposing what? Of course you need to unregister pages
eventually otherwise your pinned memory on the destination
will just grow indefinitely. People are often doing
registration caches to help reduce the overhead,
but never unregistering seems too aggressive.
You mean the chunk-based thing just delays the agony
until all guest memory is pinned for RDMA anyway?
Wait, is it registered for life on the source too?
Well this kind of explains why qemu was dying on OOM,
doesn't it?
Second, this means that my previous analysis showing that
performance was reduced
was also incorrect because most of the RDMA transfers were against
pages during
the bulk phase round, which incorrectly makes dynamic page
registration look bad.
I should have done more testing *after* the bulk phase round,
and I apologize for not doing that.
Indeed when I do such a test (with the 'stress' command) the cost of
page registration disappears
because most of the registrations have already completed a long time ago.
Thanks, Paolo for reminding us about the bulk-phase behavior to being with.
Third, this means that optimizing this protocol would not be helpful
and that we should
follow the "keep it simple" approach because during steady-state
phase of the migration
most of the pages should have already been registered.
- Michael
But if you mean to say that the current chunk based code
is useless, then I'd have to agree.