Re: [Qemu-devel] [Query] Live Migration between machines with different processor ids

[Dr. David Alan Gilbert]

* Jaggi, Manish (address@hidden) wrote:

> Just to add what happens in ARM64 case, qemu running on Machine A sends cpu 
> state information to Machine B.
> This state contains MIDR value, and so Processor ID value is compared in KVM 
> and not in qemu (correcting myself).
> 
> IIRC, Peter/Eric please point if there is something incorrect in the below 
> flow...
> 
> (Machine B)
> target/arm/machine.c: cpu_post_load()
>               - updates cpu->cpreg_values[i] : which includes MIDR (processor 
> ID register)
> 
>               - calls write_list_to_kvmstate(cpu, KVM_PUT_FULL_STATE)
> 
>                               target/arm/kvm.c: write_list_to_kvmstate
>                               - calls => kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, 
> &r);
> 
>                                       => and it eventually lands up IIRC in 
> Linux code in 
> 
>                                                       => 
> arch/arm64/kvm/sys_regs.c : set_invariant_sys_reg(u64 id, void __user *uaddr)
>                                                               /* This is what 
> we mean by invariant: you can't change it. */
>                                                               if (r->val != 
> val)
>                                                                       return 
> -EINVAL;
>                                                               Note: MIDR_EL1 
> is invariant register.
> result: Migration fails on Machine B.
> 
> A few points:
> - qemu on arm64 is invoked with -machine virt and -cpu as host. So we don't 
> explicitly define which cpu. 

Note that even on x86 we don't guarantee as much about '-cpu host', what
we expect to work for migration is that if you pick a   '-cpu amodel'
and both hosts support the feature flags required by 'amodel' then it
should work.

> - In case Machine A and Machine B have almost same Core and the delta may-not 
> have any effect on qemu operation, migration should work by just looking into 
> whitelist.
> whitelist can be given as a parameter for qemu on machine B.
> 
> qemu-system-aarch64 -whitelist <ids separated by commas>
> 
> (This is my proposal)
> 
> - So in cpu_post_load (Machine B) qemu can lookup whitelist and replace the 
> MIDR with the one at Machine B. 
> Sounds good?
> 
> - Juan raised a point about clock speed, I am not sure it will have any 
> effect on arm since qemu is run with -cpu host param.
> I could be wrong here, Peter/Eric can you please correct me...

Clock speed is only really a problem for things like timestamp counters;
some cores let you scale them; for those that don't then yes it's a bit
odd.

Dave

> -Thanks
> Manish
> 
> 
> 
> > Dave
> >> - is providing a -mirate-compat-whitelist <file> option for arm only looks 
> >> good?
> >> this option can be added in A1/A2 qemu command, so it would be upstream / 
> >> downstream agnostic.
> > 
> >>> 
> >>> I am sorry to not be able to tell you that this is an easy problem.
> >>> 
> >>> Later, Juan.
> >> 
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK