Re: [Qemu-devel] Live migration from Qemu 2.12 hosts to Qemu 3.2 hosts, with VMX flag enabled in the guest?

[Paolo Bonzini]

On 18/01/19 11:21, Daniel P. Berrangé wrote:
> On Fri, Jan 18, 2019 at 10:16:34AM +0000, Dr. David Alan Gilbert wrote:
>> * Paolo Bonzini (address@hidden) wrote:
>>> On 18/01/19 11:02, Dr. David Alan Gilbert wrote:
>>>>>
>>>>> It fails if the flag is set, rather than if any nested virtualization has
>>>>> been used before.
>>>>>
>>>>> I'm concerned I will end up with a requirement for *all* guests to be
>>>>> restarted in order to migrate them to the new hosts, rather than just the
>>>>> ones that would have a problem.
>>>> I think you should be able to migrate from 2.12->3.1 like this, but
>>>> you'd hit the problem when you then try and migrate again between your
>>>> new QEMUs.
>>>>
>>>> I guess we could modify it to wire it to machine type, so that
>>>> older machine types didn't block.
>>>
>>> That would also be wrong.  The old machine types _could_ be using KVM
>>> and they have no way to block the live migration.
>>>
>>> The solution is to restart the VM using "-cpu host,-vmx".
>>
>> The problem as Christian explained in that thread is that it was common
>> for them to start VMs with vmx enabled but for people not to use it
>> on most of the VMs, so we break migration for most VMs even though most
>> don't use it.
>>
>> It might not be robust, but it worked for a lot of people most of the
>> time.

It's not "not robust" (like, it usually works but sometimes fails
mysteriously).  It's entirely broken, you just don't notice that it is
if you're not using the feature.

> Yes, this is exactly why I said we should make the migration blocker
> be conditional on any L2 guest having been started. I vaguely recall
> someone saying there wasn't any way to detect this situation from
> QEMU though ?

You can check that and give a warning (check that CR4.VMXE=1 but no
other live migration state was transferred).  However, without live
migration support in the kernel and in QEMU you cannot start VMs *for
the entire future life of the VM* after a live migration.  So even if we
implemented that kind of blocker, it would fail even if no VM has been
started, as long as the kvm_intel module is loaded on migration.  That
would be no different in practice from what we have now.

It might work to unload the kvm_intel module and run live migration with
the CPU configured differently ("-cpu host,-vmx") on the destination.

Paolo