[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [RFC] More robust migration

From: Anthony Liguori
Subject: Re: [Qemu-devel] [RFC] More robust migration
Date: Fri, 20 Feb 2009 09:15:58 -0600
User-agent: Thunderbird (X11/20090105)

Hi Andre,

Andre Przywara wrote:

after fiddling around with migration (and the data dumped into the stream) I found the current concept possesses some shortcomings.

Yikes :-) FWIW, I focused a lot on robustness in the implementation so hopefully a lot of what you mention below were conscious decisions with very specific reasoning.

I am interested in your opinions whether it is worth to implement a new improved format.

FWIW, the format is sufficiently versioned that it isn't necessary to completely change it (not that I think it needs changing).

Issues I would like to address:
1. Transfer configuration data. Currently there is no VM configuration data transferred with the stream.

Yes, the difficulty here is that we need to transfer the machine configuration but not the host configuration. Management tools should decide how to configure the host on the target side but we should be passing the machine configuration.

If you've been following the config file threads, I've mentioned this as a use case for the current design a number of times. We would pass a flattened device tree as another savevm section with a well known name (like "machine"). Given the semantics of the current migration protocol, this would ensure that the machine generated on the remote node was exactly the same as the source node.

One has to start QEMU/KVM with the _exact_ same parameters on the other side to allow migration. If there would be a pseudo-device (transferred first) holding these parameters (and other runtime dependent stuff like kvm_enabled()) this would ease migration a lot.

FWIW, there's nothing preventing migrating from TCG -> KVM.

I think one can debate about whether host config should be migrated too. I'd argue that in the core migration protocol, host config should not be present. I think you can have an easier to use migration protocol (like the old ssh protocol) that also transferred host config. But in the general case, you want management tools to be able to manipulate host config upon migration.

2. Introduce a length field to the header of each device.

IMHO, this would reduce robustness. It's also difficult because of the way savevm registration works. You don't know how large a section is until it's written and migration streams are not seekable.

This would allow to skip unknown (or unwanted) devices.

No good can come from this. If you have an unknown section, you must throw and error and stop the migration. What if this is for a device that the guest is interacting with? The device just disappears after migration? All savevm state is state that affects the functionality of a guest. Throwing away this state will change the functionality of the VM and migration should not affect guest functionality.

I know this imposes a bit of a challenge, because the length is not always known in advance, but one could overcome this (by using the buffer to patch in the length later for instance).

What are the use cases where you think this would be beneficial? I really see the change in semantics from the old way (throwing away unknown sections) to the new way (requiring strict versioning and validating all sections) as being a huge step toward robustness.

3. Make the device versioning really bulletproof. Currently some devices dump different data depending on runtime (or better time-of-creation) state (for instance hw/i8254.c: if (s->irq_timer)...).

If you look carefully, s->irq_timer will always be set. The checks are unnecessary.

Another example is the (x86?) CPU state, which differs with KVM en/disabled.

Not in upstream QEMU...

Some devices even dump host system dependent structures (like struct vecio in virtio-blk.c).

That is awful and needs to be fixed. It should have never been committed like that.

Also one could create some kind of (limited) upward compatibility, so older QEMU versions ignore additional, but optional fields in a device state (similar to the ext2 compatibility scheme). Maybe this could be done by an external converter program.

To me, ignoring is always a bad thing. It's almost always going to be unsafe. Doesn't this decrease robustness by being less conservative?

4. Allow optional devices. Some devices are always started (like HPET), although they don't need to be used by the OS. If one migrates such a guest from say KVM-83 to KVM-81, it will fail, because KVM-81 does not support HPET. One could migrate the device only if it has been used.

There's no way you can migrate from KVM-83 to KVM-81 if you've enabled the HPET. It cannot be made to work.

There is a -no-hpet option though. If you are a management tool that needs to support migration from multiple versions, you should use -no-hpet. Also, if you need to migrate from KVM-81 to KVM-83, you should use -no-hpet with KVM-83 to avoid changing the guest visible state.

In the long run, the machine configuration file will address this in a more thorough manner. FWIW, -no-hpet was added specifically to deal with migration.

In general I would like to know whether QEMU migration is intended to be used in such a flexible manner or whether the requirement of the exact same software version on both side is not a limitation in everyday use.

My primary goal for migration is robustness. I do not think it's a good idea to support any circumstances that could introduce changes in guest visible state during a live migration.

Live migration is a critical feature for many production environments. To be useful IMHO, it has to be bullet-proof.


Anthony Liguori

Awaiting your comments!


reply via email to

[Prev in Thread] Current Thread [Next in Thread]