Re: [Qemu-devel] [RFC 0/6] enable numa configuration before machine_init() from HMP/QMP

[Igor Mammedov]

On Mon, 16 Oct 2017 17:36:36 +0100
"Daniel P. Berrange" <address@hidden> wrote:

> On Mon, Oct 16, 2017 at 06:22:50PM +0200, Igor Mammedov wrote:
> > Series allows to configure NUMA mapping at runtime using QMP/HMP
> > interface. For that to happen it introduces a new '-paused' CLI option
> > which allows to pause QEMU before machine_init() is run and
> > adds new set-numa-node HMP/QMP commands which in conjuction with
> > info hotpluggable-cpus/query-hotpluggable-cpus allow to configure
> > NUMA mapping for cpus.  
> 
> What's the problem we're seeking solve here compared to what we currently
> do for NUMA configuration ?
From RHBZ1382425
"
Current -numa CLI interface is quite limited in terms that allow map CPUs to 
NUMA nodes as it requires to provide cpu_index values which are non obvious and 
depend on machine/arch. As result libvirt has to assume/re-implement cpu_index 
allocation logic to provide valid values for -numa cpus=... QEMU CLI option.

Now QEMU has in place generic CPU hotplug interface and ability to query 
possible CPUs layout (with QMP command query-hotpluggable-cpus),
however it requires to run QEMU once per each machine type and topology 
configuration (-M & -smp combination) which would be too taxing for mgmt layer 
to do.
Currently proposed idea to solve the issue is to do NUMA mapping at runtime:
 1. start QEMU in stopped mode with needed -M & -smp configuration
    but leave out "-numa cpus" options
 2. query possible cpus layout (query-hotpluggable-cpus)
 3. use new QMP command to map CPUs to NUMA node in terms of generic CPU
    hotplug interface (socket/core/thread)

    commit (419fcde numa: add '-numa cpu,...' option for property based node 
mapping)
    added CLI option for topology based

...
 4. continue VM exection
"

> > 
> > HMP configuration session for CLI '-smp 1,maxcpus=2' would look like:
> > 
> > (qemu) info hotpluggable-cpus 
> > Hotpluggable CPUs:
> >   type: "qemu64-x86_64-cpu"
> >   vcpus_count: "1"
> >   CPUInstance Properties:
> >     socket-id: "1"
> >     core-id: "0"
> >     thread-id: "0"
> >   type: "qemu64-x86_64-cpu"
> >   vcpus_count: "1"
> >   qom_path: "/machine/unattached/device[0]"
> >   CPUInstance Properties:
> >     socket-id: "0"
> >     core-id: "0"
> >     thread-id: "0"
> > (qemu) set-numa-node node,nodeid=0
> > (qemu) set-numa-node node,nodeid=1
> > (qemu) set-numa-node cpu,socket-id=0,node-id=0
> > (qemu) set-numa-node cpu,socket-id=1,node-id=1
> > (qemu) info hotpluggable-cpus 
> > Hotpluggable CPUs:
> >   type: "qemu64-x86_64-cpu"
> >   vcpus_count: "1"
> >   CPUInstance Properties:
> >     node-id: "1"
> >     socket-id: "1"
> >     core-id: "0"
> >     thread-id: "0"
> >   type: "qemu64-x86_64-cpu"
> >   vcpus_count: "1"
> >   CPUInstance Properties:
> >     node-id: "0"
> >     socket-id: "0"
> >     core-id: "0"
> >     thread-id: "0"
> > (qemu) cont
> > 
> > git tree for testing:
> >   https://github.com/imammedo/qemu qmp_preconfig_rfc
> > 
> > 
> > CC: address@hidden
> > CC: address@hidden
> > CC: address@hidden
> > CC: address@hidden
> > CC: address@hidden
> > CC: address@hidden
> > CC: address@hidden
> > CC: address@hidden
> > 
> > Igor Mammedov (6):
> >   numa: postpone options post-processing till machine_run_board_init()
> >   numa: split out NumaOptions parsing into parse_NumaOptions()
> >   possible_cpus: add CPUArchId::type field
> >   CLI: add -paused option
> >   HMP: add set-numa-node command
> >   QMP: add set-numa-node command
> > 
> >  hmp.h                      |  1 +
> >  include/hw/boards.h        |  2 ++
> >  include/sysemu/numa.h      |  2 ++
> >  include/sysemu/sysemu.h    |  1 +
> >  hmp-commands.hx            | 13 ++++++++
> >  hmp.c                      | 23 ++++++++++++++
> >  hw/arm/virt.c              |  3 +-
> >  hw/core/machine.c          | 18 ++++++-----
> >  hw/i386/pc.c               |  4 ++-
> >  hw/ppc/spapr.c             | 13 +++++---
> >  hw/s390x/s390-virtio-ccw.c |  1 +
> >  numa.c                     | 79 
> > ++++++++++++++++++++++++++++++++++------------
> >  qapi-schema.json           | 13 ++++++++
> >  qemu-options.hx            | 15 +++++++++
> >  qmp.c                      |  5 +++
> >  vl.c                       | 54 ++++++++++++++++++++++++++++++-
> >  16 files changed, 210 insertions(+), 37 deletions(-)
> > 
> > -- 
> > 2.7.4
> > 
> >   
> 
> Regards,
> Daniel