Re: [Qemu-devel] [RFC v2 0/6] QEMU shared-memory backend

[Baptiste Reynal]

On Tue, Mar 22, 2016 at 3:14 PM, Markus Armbruster <address@hidden> wrote:
>
> Copying Paolo, author of memory-backend-file.
>
> Baptiste Reynal <address@hidden> writes:
>
> > A new memory backend, the shared memory backend, based on
> > the file memory backend.
> >
> > This new backend allows a master QEMU instance to share a part of
> > his main memory whith a slave QEMU instance. It is then possible to load
> > a firmware on this memory and trigger the slave boot using a SDM
> > signal.
>
> Can you explain why that's useful in a bit more detail?
>

The purpose of such memory backend is to enable the modeling
of asymmetric multiprocessing (AMP) configurations. An example is
a multi-core ARM CPU working alongside with two Cortex-M
micro controllers.

On such configurations many CPU (modeled by many QEMU
instances) share the same memory bus. This patch series allows multiple
QEMU instances to share the same file-backed memory bus.

>
> > Such new backend enables, on a master side, to allocate the whole
> > memory as shareable (e.g. /dev/shm, or hugetlbfs).
>
> memory-backend-file can do that already.  What exactly does
> memory-backend-shared add to it?

This memory-backend-shared allows multiple QEMU instances to share the
same memory backend. A slave instance will have access only to a subset of
the whole AMP system memory, by mapping just one part of the file as its
memory. This can be considered as an MMU configured by the master processor
to restrict slave memory accesses to only a subset of the whole system memory.

For the purpose of AMP modeling, a QEMU instance can be put in waiting mode
(by using QEMU migration) until a master QEMU triggers its boot. It allows the
master to fill the slave memory with its firmware before boot, in order to be
compliant with the remoteproc framework (power on, load firmware, power off).

>
>
> > On the slave side it enables the startup of QEMU without any main memory
> > allocated. Then the slave goes in a waiting state, the same used in the
> > case of an incoming migration, and the file descriptor of the shared
> > memory is sent by the master over a socket.
> > The waiting state ends when the master sends to the slave a signal
> > with size and offset to mmap and use as memory as payload.
> >
> > You can test those patches using modules and test application available
> > on https://git.virtualopensystems.com/dev/qemu-het-tools.
> > - SDM Platform module on branch sdm_test_mod_v1
> > - Test application on branch sdm_test_app
> > - Slave files on branch demo_slaves_sdm
> >
> >  QEMU code is avalaible here https://git.virtualopensystems.com/dev/qemu on 
> > branch rfc_v2.
> >
> > Run QEMU master instance:
> > ./qemu-system-arm -nographic \
> >         -kernel zImage \
> >         -M virt -m 1G \
> >         -initrd busybox.cpio \
> >         -object multi-socket-backend,id=sock,path=sock,listen \
> >         -object 
> > memory-backend-shared,id=mem,size=1G,mem-path=/dev/hugepages,socket=sock,master
> >  \
> >         -object sdm-signal-shboot,id=boot,shm=mem \
> >         -object sdm-communication-socket,id=scomm,socket=sock \
> >         -device 
> > virtio-sdm-device,comm=scomm,master,num-slaves=1,len-signals=1,signals[0]=boot
> >  \
> >         -append "mem=512M memmap=512M$0x60000000" \
> >         -numa node,memdev=mem -m 1G
> >
> > Run QEMU slave instance:
> > ./qemu-system-arm -nographic \
> >         -kernel zImage \
> >         -M virt -m 1G \
> >         -initrd busybox.cpio \
> >         -object multi-socket-backend,id=sock,path=sock \
> >         -object memory-backend-shared,id=mem,size=512M,socket=sock \
> >         -object sdm-signal-shboot,id=boot,shm=mem \
> >         -object sdm-communication-socket,id=scomm,socket=sock \
> >         -device sdm-platform,comm=scomm,len-signals=1,signals[0]="boot" \
> >         -incoming "shared:mem" \
> >         -numa node,memdev=mem -m 512M
> >
> > On the master, load the SDM module (insmod sdm_test_mod_v1) and trigger the 
> > boot with
> > the following command:
> > ./sdm-test -b 0 1 zImage slave.dtb
> >
> > zImage and slave.dtb are available on branch demo_slaves_sdm.
> >
> > This patch serie is a follow-up to "[RFC PATCH 0/8] Towards an 
> > Heterogeneous QEMU":
> > https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg00171.html
> >
> > This patch serie depends on:
> > - [RFC v2 0/6] SDM Interface
> > - [RFC v2 1/1] backend: multi-client-socket
> >
> > This work has been sponsored by Huawei Technologies Duesseldorf GmbH.
> >
> > Baptiste Reynal (3):
> >   backend: shared memory backend
> >   migration: add shared migration type
> >   hw/misc: sdm signal shboot
> >
> > Christian Pinto (3):
> >   qemu: slave machine flag
> >   hw/arm: boot
> >   qemu: numa
> >
> >  backends/Makefile.objs              |   2 +-
> >  backends/hostmem-shared.c           | 198 
> > ++++++++++++++++++++++++++++++++++++
> >  hw/arm/boot.c                       |  13 +++
> >  hw/core/machine.c                   |  27 +++++
> >  hw/misc/Makefile.objs               |   1 +
> >  hw/misc/sdm-signal-shboot.c         |  62 +++++++++++
> >  include/hw/boards.h                 |   2 +
> >  include/hw/misc/sdm-signal-shboot.h |  38 +++++++
> >  include/migration/migration.h       |   2 +
> >  include/sysemu/hostmem-shared.h     |  66 ++++++++++++
> >  migration/Makefile.objs             |   2 +-
> >  migration/migration.c               |   2 +
> >  migration/shared.c                  |  33 ++++++
> >  numa.c                              |  17 +++-
> >  qemu-options.hx                     |   5 +-
> >  util/qemu-config.c                  |   5 +
> >  16 files changed, 471 insertions(+), 4 deletions(-)
> >  create mode 100644 backends/hostmem-shared.c
> >  create mode 100644 hw/misc/sdm-signal-shboot.c
> >  create mode 100644 include/hw/misc/sdm-signal-shboot.h
> >  create mode 100644 include/sysemu/hostmem-shared.h
> >  create mode 100644 migration/shared.c