Re: [PATCH RFC V3 17/29] arm/virt: Release objects for *disabled* possible vCPUs after init

[Gavin Shan]

Hi Salil,

On 8/21/24 2:40 AM, Salil Mehta wrote:
> I don’t understand this clearly. Are  you suggesting to reuse only single
> vCPU object to initialize all KVM vCPUs not yet plugged? If yes, then
> I'm not sure what do we gain here by adding this complexity? It does
> not consume time or resources because we are not realizing any of these
> vCPU object in any case.

First of all, it seems we have different names and terms for those cold-booted 
vCPUs
and hotpluggable vCPUs. For example, vCPU-0 and vCPU-1 are cold-booted vCPUs 
while
vCPU-2 and vCPU-3 are hotpluggable vCPUs when we have '-smp maxcpus=4,cpus=2'. 
Lets
stick to convention and terms for easier discussion.

The idea is avoid instantiating hotpluggable vCPUs in virtmach_init() and 
released in the
same function for those hotpluggable vCPUs. As I can understand, those 
hotpluggable vCPU
instances are serving for two purposes: (1) Relax the constraint that all 
vCPU's (kvm)
file descriptor have to be created and populated; (2) Help to instantiate and 
realize
GICv3 object.

For (1), I don't think we have to instantiate those hotpluggable vCPUs at all. 
In the above
example where we have command line '-smp maxcpus=4,cpus=2', it's unnecessary to 
instantiate
vCPU-3 and vCPU-4 to create and populate their KVM file descriptors. A vCPU's 
KVM file
descriptor is create and populated by the following ioctls and function calls. 
When the first
vCPU (vCPU-0) is realized, the property corresponding to "&init" is fixed for 
all vCPUs. It
means all vCPUs have same properties except the "vcpu_index".

  ioctl(vm-fd,   KVM_CREATE_VCPU,   vcpu_index);
  ioctl(vcpu-fd, KVM_ARM_VCPU_INIT, &init);
  kvm_park_vcpu(cs);

A vCPU's properties are determined by two sources and both are global. It means 
all vCPUs
should have same properties: (a) Feature registers returned from the host. The 
function
kvm_arm_get_host_cpu_features() is called for once, meaning this source is same 
to all vCPUs;
(b) The parameters provided by user through '-cpu host,sve=off' are translated 
to global
properties and applied to all vCPUs when they're instantiated.

      (a)                                            (b)

  aarch64_host_initfn                          qemu_init
  kvm_arm_set_cpu_features_from_host           parse_cpu_option
    kvm_arm_get_host_cpu_features              cpu_common_parse_features
                                               qdev_prop_register_global
                                                 :
                                               device_post_init
                                               qdev_prop_set_globals

For (2), I'm still looking into the GICv3 code for better understanding. Until 
now, I don't
see we need the instantiated hotpluggable vCPUs either. For example, the 
redistributor regions
can be exposed based on 'maxcpus' instead of 'cpus'. The IRQ connection and 
teardown can be
dynamically done by connecting the board with GICv3 through callbacks in 
ARMGICv3CommonClass.
The connection between GICv3CPUState and CPUARMState also can be done 
dynamically.

Thanks,
Gavin