Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Mon, 2015-01-26 at 12:27 +0100, Alexandre DERUMIER wrote:
> >>About this, I can do it successfully on my qemu. 
> >>So can you tell us more information about your operation? 
> simply start with
> 
> -smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2
> 
> then
> 
> #device_del cpu2
> 
> Guest return
> [  324.195024] Unregister pv shared memory for cpu 2
> [  324.250579] smpboot: CPU 2 is now offline
> 
> but cpu is not remove in qemu.
> 
> I had also try to eject manualy from guest
> echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject
> 
> But I have this error message
> ACPI: \_SB_.CP02: Eject incomplete - status 0xf
> 
> (maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian 
> wheezy backports) ?)
> 

According to your description, the guest has invoked _EJ0 method, but
the method was executed unsuccessfully.
Thus, only hot-unplug request was invoked in qemu, but hot-unplug was
not so that cpu was still in qemu.

Did you test any other kernel version? I will try kernel 3.16 to
investigate this problem later.

Regards,
Zhu

> 
> >>And I found the patchset you applied is the last version in your last 
> >>email. I think you'd better apply the latest version. 
> 
> I have applied on top of git theses patches series
> 
> https://www.mail-archive.com/qemu-devel%40nongnu.org/msg272745.html
> [Qemu-devel] [RESEND PATCH v1 0/5] Common unplug and unplug request cb for 
> memory and CPU hot-unplug.
> https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
> [Qemu-devel] [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu        support
> https://www.mail-archive.com/address@hidden/msg273870.html
> [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> ----- Mail original -----
> De: "Zhu Guihua" <address@hidden>
> À: "aderumier" <address@hidden>
> Cc: "qemu-devel" <address@hidden>, address@hidden, "guz fnst" 
> <address@hidden>, "isimatu yasuaki" <address@hidden>, "Anshul Makkar" 
> <address@hidden>, "chen fan fnst" <address@hidden>, "Igor Mammedov" 
> <address@hidden>, "afaerber" <address@hidden>
> Envoyé: Lundi 26 Janvier 2015 04:47:13
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> On Mon, 2015-01-26 at 04:19 +0100, Alexandre DERUMIER wrote: 
> > Thanks for your reply. 
> > 
> > 2 others things: 
> > 
> > 1) 
> > on cpu unplug, I see that the cpu is correctly removed from my linux guest 
> > but not from qemu 
> > 
> 
> About this, I can do it successfully on my qemu. 
> So can you tell us more information about your operation? 
> 
> And I found the patchset you applied is the last version in your last 
> email. I think you'd better apply the latest version. 
> 
> Regards, 
> Zhu 
> 
> > starting with a guest with 3cpus: 
> > 
> > guest: #ls -lah /sys/devices/system/ |grep cpu 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2 
> > 
> > hmp: # info cpus 
> > * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> > CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> > CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> > 
> > 
> > then unplug cpu2 
> > hmp : device_del cpu2 
> > 
> > guest: 
> > 
> > dmesg: 
> > [ 176.219754] Unregister pv shared memory for cpu 2 
> > [ 176.278881] smpboot: CPU 2 is now offline 
> > 
> > #ls -lah /sys/devices/system/ |grep cpu 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> > 
> > hmp: # info cpus 
> > * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> > CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> > CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> > 
> > 
> > 
> > 
> > 2)when numa is used, the hotplugged cpu is always on numa node 0 
> > (cpu_add or device_add cpu) 
> > 
> > 
> > starting a guest, with 2 sockets,1 cores 
> > 
> > -smp 2,sockets=2,cores=1,maxcpus=2 
> > -object memory-backend-ram,size=256M,id=ram-node0 -numa 
> > node,nodeid=0,cpus=0,memdev=ram-node0 
> > -object memory-backend-ram,size=256M,id=ram-node1 -numa 
> > node,nodeid=1,cpus=1,memdev=ram-node1 
> > 
> > hmp: 
> > # info numa 
> > 2 nodes 
> > node 0 cpus: 0 
> > node 0 size: 256 MB 
> > node 1 cpus: 1 
> > node 1 size: 256 MB 
> > 
> > ok 
> > 
> > now 
> > 
> > starting with same topology, but with 1cpu at start 
> > -smp 2,sockets=2,cores=1,maxcpus=2 
> > -object memory-backend-ram,size=256M,id=ram-node0 -numa 
> > node,nodeid=0,cpus=0,memdev=ram-node0 
> > -object memory-backend-ram,size=256M,id=ram-node1 -numa 
> > node,nodeid=1,cpus=1,memdev=ram-node1 
> > 
> > # info numa 
> > 2 nodes 
> > node 0 cpus: 0 
> > node 0 size: 256 MB 
> > node 1 cpus: 
> > node 1 size: 256 MB 
> > 
> > hotpluging a cpu 
> > # device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > 
> > # info numa 
> > 2 nodes 
> > node 0 cpus: 0 1 
> > node 0 size: 256 MB 
> > node 1 cpus: 
> > node 1 size: 256 MB 
> > 
> > cpu1 should be on node1, not node0. 
> > 
> > 
> > Regards, 
> > 
> > Alexandre 
> > 
> > ----- Mail original ----- 
> > De: "Zhu Guihua" <address@hidden> 
> > À: "aderumier" <address@hidden> 
> > Cc: "qemu-devel" <address@hidden>, address@hidden, "guz fnst" 
> > <address@hidden>, "isimatu yasuaki" <address@hidden>, "Anshul Makkar" 
> > <address@hidden>, "chen fan fnst" <address@hidden>, "Igor Mammedov" 
> > <address@hidden>, "afaerber" <address@hidden> 
> > Envoyé: Lundi 26 Janvier 2015 03:01:48 
> > Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove 
> > support 
> > 
> > On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> > > Hello, 
> > > 
> > > I'm currently testing the new cpu unplug features, 
> > > Works fine here with debian guests and kernel 3.14. 
> > > 
> > 
> > Thanks for your test. 
> > 
> > > But I have notice some small potential bugs, but I'm not sure I'm doing 
> > > it right. 
> > > 
> > > 1)first, to unplug cpu, we need an id for cpu 
> > > 
> > 
> > Yes, if you want to unplug cpu, you must have an id for cpu. 
> > 
> > > The problem is that the current qemu command line 
> > > -smp 1,sockets=2,cores=1,maxcpus=2 
> > > 
> > > for example, will create 1 cpu on apic-id 0 without any id, so we can't 
> > > unplug it. 
> > > 
> > > 
> > > So, I have tried with 
> > > 
> > > -smp 1,sockets=2,cores=1,maxcpus=2 -device 
> > > kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> > > 
> > > But this give me an error: 
> > > "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> > > 
> > 
> > APIC ID 0 was used by the cpu of '-smp 1'. 
> > So you should use apic-id=1 
> > 
> > > (also try to set -smp 0, but it's not working). 
> > > 
> > > 
> > > 
> > > 2) second problem, if I start with 
> > > -smp 1,sockets=2,cores=1,maxcpus=2 
> > > 
> > > then hmp: 
> > > device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > > 
> > > then hmp : device_del cpu1 
> > > 
> > > Got an error:" 
> > > This is the last cpu, should not be removed!" 
> > > 
> > > 
> > 
> > Oh, it's our problem, thanks for your pointing out. 
> > I will fix it in next version. 
> > 
> > Regards, 
> > Zhu 
> > 
> > > 
> > > This is coming from 
> > > [PATCH 06/12] pc: add cpu hot unplug request callback support 
> > > + if (smp_cpus == 1) { 
> > > + error_setg(&local_err, 
> > > + "This is the last cpu, should not be removed!"); 
> > > + goto out; 
> > > + } 
> > > 
> > > 
> > > 
> > > So, the only way unplug is working for me, is to start with -smp 2 
> > > minimum 
> > > -smp 2,sockets=2,cores=1,maxcpus=4 
> > > 
> > > Then I can hotplug|unplug cpuid >= 2 
> > > 
> > > 
> > > 
> > > Regards, 
> > > 
> > > Alexandre Derumier 
> > [...]