[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

From: Anthony Liguori
Subject: Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy
Date: Wed, 02 Mar 2011 07:39:01 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20101027 Lightning/1.0b1 Thunderbird/3.0.10

On 03/01/2011 03:59 AM, Dor Laor wrote:
On 02/28/2011 08:12 PM, Anthony Liguori wrote:

On Feb 28, 2011 11:47 AM, "Avi Kivity" <address@hidden
<mailto:address@hidden>> wrote:
> On 02/28/2011 07:33 PM, Anthony Liguori wrote:
>> >
>> > You're just ignoring what I've written.
>> No, you're just impervious to my subtle attempt to refocus the
discussion on solving a practical problem.
>> There's a lot of good, reasonably straight forward changes we can
make that have a high return on investment.
> Is making qemu the authoritative source of configuration information
a straightforward change? Is the return on it high? Is the investment low?

I think this is where we fundamentally disagree.  My position is that
QEMU is already the authoritative source.  Having a state file doesn't
change anything.

Do a hot unplug of a network device with upstream libvirt with acpiphp
unloaded, consult libvirt and then consult the monitor to see who has
the right view of the guests config.

To me, that's the definition of authoritative.

> "No" to all three (ignoring for the moment whether it is good or not,
which we were debating).
>> The only suggestion I'm making beyond Marcelo's original patch is
that we use a structured format and that we make it possible to use the
same file to solve this problem in multiple places.
> No, you're suggesting a lot more than that.

That's exactly what I'm suggesting from a technical perspective.

>> I don't think this creates a fundamental break in how management
tools interact with QEMU.  I don't think introducing RAID support in the
block layer is a reasonable alternative.
> Why not?

Because its a lot of complexity and code that can go wrong while only
solving the race for one specific case.  Not to mention that we double
the iop rate.

> Something that avoids the whole state thing altogether:
> - instead of atomically switching when live copy is done, keep on
issuing writes to both the origin and the live copy
> - issue a notification to management
> - management receives the notification, and issues an atomic blockdev
switch command

> this is really the RAID-1 solution but without the state file (credit
Dor).  An advantage is that there is no additional latency when trying
to catch up to the dirty bitmap.

It still suffers from the two generals problem.  You cannot solve this
without making one node reliable and that takes us back to it being
either QEMU (posted event and state file) or the management tool (sync

It is safe w/o a state file by changing the basic live copy algorithm:

1. Live copy in progress stage
   Once live copy command is issued, a dirty bit map is created for
   tracking. There is a single pass over the entire image where we copy
   blocks from the src to the dst.

   Write commands for blocks that were already copied will be done
   twice for the src and dst.

   Once the full copy single pass ends, we trigger a QMP event that
   this stage can end.

   The live copy stage keeps running till the management issue a switch
   command. When it will happen, the switch is immediate and no need to
   copy additional blocks (but flush pending IOs).

2. Management sends a switch command.
   Qemu stops the doubling the IO and switches to the destination.

Here is where your race is:

2. Management sends a switch command

3. QEMU receives switch command

4. QEMU stops doubling IO and switches to the destination

5. QEMU sends acknowledgement of switch command

6. Management receives acknowledge of switch command

7. Management changes internal state definition to reflect the new destination

If QEMU or the management tool crashes after step 4 and before step 6, when the management tool restarts QEMU with the source image, data loss will have occurred (and potentially corruption if a flush had happened).

This all boils down to the Two Generals Problem[1]. It's simply not fixable without making one end reliable and that means that someone needs to fsync() something *after* the switchover happens but before the first write happens. That can be QEMU (Avi's RAID proposal and my state file proposal) or it can be the management tool (if we introduce synchronous events).

[1] http://en.wikipedia.org/wiki/Two_Generals%27_Problem


Anthony Liguori

reply via email to

[Prev in Thread] Current Thread [Next in Thread]