|
From: | Anthony Liguori |
Subject: | Re: [Qemu-devel] [RFC] postcopy livemigration proposal |
Date: | Mon, 08 Aug 2011 10:29:03 -0500 |
User-agent: | Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Lightning/1.0b2 Thunderbird/3.1.10 |
On 08/08/2011 10:11 AM, Dor Laor wrote:
On 08/08/2011 03:32 PM, Anthony Liguori wrote:On 08/08/2011 04:20 AM, Dor Laor wrote:That's terrific (nice video also)! Orit and myself had the exact same idea too (now we can't patent it..). Advantages: - No down time due to memory copying.But non-deterministic down time due to network latency while trying to satisfy a page fault.True but it is possible to limit it with some dedicated network or bandwidth reservation.
Yup. Any technique that uses RDMA (which is basically what this is) requires dedicated network resources.
- Efficient, reduce needed traffic no need to re-send pages.It's not quite that simple. Post-copy needs to introduce a protocol capable of requesting pages.Just another subsection.. (kidding), still it shouldn't be too complicated, just an offset+pagesize and return page_content/error
What I meant by this is that there is potentially a lot of round trip overhead. Pre-copy migration works well with reasonable high latency network connections because the downtime is capped only by the maximum latency sending from one point to another.
But with something like this, the total downtime is 2*max_latency*nb_pagefaults. That's potentially pretty high.
So it may be desirable to try to reduce nb_pagefaults by prefaulting in pages, etc. Suffice to say, this ends up getting complicated and may end up burning network traffic too.
This is really just a limitation of our implementation. In theory, pre-copy allows you to exert fine grain resource control over the guest which you can use to encourage convergence.But a very large guest w/ large working set that changes more frequent than the network bandwidth might always need huge down time with the current system.
In theory, you can do things like reduce the guests' priority to reduce the amount of work it can do in order to encourage convergence.
One thing I think we need to do is put together a live migration roadmap. We've got a lot of invasive efforts underway with live migration and I fear that without some planning and serialization, some of this useful work with get lost.Some of them are parallel. I think all the readers here agree that post copy migration should be an option while we need to maintain the current one.
I actually think they need to be done mostly in sequence while cleaning up some of the current infrastructure. I don't think we really should make any major changes (beyond maybe the separate thread) until we eliminate QEMUFile.
There's so much overhead involved in using QEMUFile today, I think it's hard to talk about performance data when we've got a major bottleneck sitting in the middle.
Regards, Anthony Liguori
[Prev in Thread] | Current Thread | [Next in Thread] |