Re: [Qemu-devel] [Query] Live Migration between machines with different processor ids

[Andrew Jones]

On Wed, Sep 05, 2018 at 12:42:05PM +0000, Jaggi, Manish wrote:
> 
> 
> > On 05-Sep-2018, at 5:50 PM, Andrew Jones <address@hidden> wrote:
> > 
> > External Email
> > 
> > On Wed, Sep 05, 2018 at 11:46:11AM +0000, Jaggi, Manish wrote:
> >> (a) Changes in KVM:
> >> 
> >> - Introducing a specific error code (KVM_EINVARIANT) in case of invariant 
> >> writes.
> >> This should not change anything to API SET_ONE_REG KVM API.
> >> Not sure which is the best place to put the define…
> >> I have added in include/uapi/linux/kvm_para.h.
> >> 
> >> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> >> index 22fbbdbece3c..c8a4fbe8a8bb 100644
> >> --- a/arch/arm64/kvm/sys_regs.c
> >> +++ b/arch/arm64/kvm/sys_regs.c
> >> @@ -1111,7 +1111,7 @@ static int __set_id_reg(const struct sys_reg_desc 
> >> *rd, void __user *uaddr,
> >> 
> >>        /* This is what we mean by invariant: you can't change it. */
> >>        if (val != read_id_reg(rd, raz))
> >> -               return -EINVAL;
> >> +               return -KVM_EINVARIANT;
> >> 
> >>        return 0;
> >> }
> >> diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
> >> index 6c0ce49931e5..3a49a321d0df 100644
> >> --- a/include/uapi/linux/kvm_para.h
> >> +++ b/include/uapi/linux/kvm_para.h
> >> @@ -17,6 +17,7 @@
> >> #define KVM_E2BIG              E2BIG
> >> #define KVM_EPERM              EPERM
> >> #define KVM_EOPNOTSUPP         95
> >> +#define KVM_EINVARIANT          96
> >> 
> >> #define KVM_HC_VAPIC_POLL_IRQ          1
> >> #define KVM_HC_MMU_OP                  2
> >> 
> >> (b) Changes in Qemu code
> >> 
> >> 1. Handling of new error code, which would update
> >> guest state with hosts invariant reg values.
> >> 
> >> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> >> index 65f867d569..0cf14323a2 100644
> >> --- a/target/arm/kvm.c
> >> +++ b/target/arm/kvm.c
> >> @@ -452,7 +452,15 @@ bool write_list_to_kvmstate(ARMCPU *cpu, int level)
> >>             abort();
> >>         }
> >>         ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
> >> -        if (ret) {
> >> +        if (ignore_invariant && (ret == -KVM_EINVARIANT)) {
> >> +            /* Update Guest invariant to match with migrated host regs*/
> >> +            ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
> >> +            if (ret)
> >> +                ok = false;
> >> +            else
> >> +                cpu->cpreg_values[i] = r.addr;
> >> +        }
> >> +        else if (ret) {
> >>             /* We might fail for "unknown register" and also for
> >>              * "you tried to set a register which is constant with
> >>              * a different value from what it actually contains".
> >> 
> >> 2. ignore_invariant is the flag I was referring to which is what you also
> >> mentioned opt-in. This can be supplied as a command line parameter to qemu 
> >> on Machine B.
> > 
> > The same libvirt folk also suggest that a QMP command be provided that
> > allows the selection of this ignore-invariant mode (in addition to or
> > instead of the command line parameter.) Otherwise a guest that has
> > already started without the parameter will not be able to migrate to
> > a "close enough” host
> 
> ignore_invariant: This can be supplied as a command line parameter to qemu on 
> Machine B.
> Machine A VM has to be migrated to B.

OK, I just confirmed with libvirt people that there's no need for the QMP
command in this case, as the source host doesn't require any knowledge of
it. The command line parameter (machine property?) must be discoverable
by libvirt, though, so it can know whether or not the destination host's
QEMU supports it.

Thanks,
drew