Re: [PATCH v14 00/21] Initial support for multi-process Qemu

[Elena Ufimtseva]

On Thu, Dec 17, 2020 at 07:57:47PM -0800, elena.ufimtseva@oracle.com wrote:
> From: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> 
> Hi
>

For some reason the patchset was not sent fully and only few patches
made trough.
I am resending the series with prefix in the subject "[PATCH v14 RESEND... "

I am sorry for this inconvenience.

Elena 
> This is the v14 of the patchset. Thank you very much for reviewing v13 and
> sharing your feedback.
> 
> We have addressed all the comments from the v13 review with changelog below:
> 
> - [PATCH v14 05/21] multi-process: Add config option for multi-process QEMU
>   add config option to disable the feature (--disable-multiprocess).
> 
> - [PATCH v14 08/21] multi-process: add qio channel write function
> - [PATCH v14 09/21] multi-process: add qio channel read function
>   refactor code in qio channel and split in two for read and write.
> 
> - [PATCH v14 04/21] socket: export socket_get_fd() function
>   export socket_get_fd() and use in remote process with some
>   modifications.
> 
> In addition, we will be sending a follow-up patch with the acceptance
> test that can be used to verify that this feature is working correctly.
> 
> From the v13 review few items are TODOs:
> - Prefix log messages with PID in the logging subsystem.
> - Refactor the MemoryListener code in vfio-user and multi-process code
>   to avoid logic duplication.
> 
> To touch upon the history of this project, we posted the Proof Of Concept
> patches before the BoF session in 2018. Subsequently, we have posted 13 
> versions on the qemu-devel mailing list. You can find them by following the
> links below ([1] - [13]). Following people contributed to the design and
> implementation of this project:
> Jagannathan Raman <jag.raman@oracle.com>
> Elena Ufimtseva <elena.ufimtseva@oracle.com>
> John G Johnson <john.g.johnson@oracle.com>
> Stefan Hajnoczi <stefanha@redhat.com>
> Konrad Wilk <konrad.wilk@oracle.com>
> Kanth Ghatraju <kanth.ghatraju@oracle.com>
> 
> We would like to thank the QEMU community for your feedback in the
> design and implementation of this project. Qemu wiki page:
> https://wiki.qemu.org/Features/MultiProcessQEMU
> 
> For the full concept writeup about QEMU multi-process, please
> refer to docs/devel/qemu-multiprocess.rst. Also, see
> docs/qemu-multiprocess.txt for usage information.
> 
> Thank you for reviewing these series.
> 
> [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
> [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
> [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
> [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
> [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
> [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html
> [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html
> [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/
> [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html
> [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html
> [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html
> [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html
> [12]: cover.1606853298.git.jag.raman@oracle.com/">https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/
> [13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html
> 
> Elena Ufimtseva (8):
>   multi-process: add configure and usage information
>   multi-process: add qio channel write function
>   multi-process: add qio channel read function
>   multi-process: define MPQemuMsg format and transmission functions
>   multi-process: introduce proxy object
>   multi-process: add proxy communication functions
>   multi-process: Forward PCI config space acceses to the remote process
>   multi-process: perform device reset in the remote process
> 
> Jagannathan Raman (12):
>   memory: alloc RAM from file at offset
>   socket: export socket_get_fd() function
>   multi-process: Add config option for multi-process QEMU
>   multi-process: setup PCI host bridge for remote device
>   multi-process: setup a machine object for remote device process
>   multi-process: Initialize message handler in remote device
>   multi-process: Associate fd of a PCIDevice with its object
>   multi-process: setup memory manager for remote device
>   multi-process: PCI BAR read/write handling for proxy & remote
>     endpoints
>   multi-process: Synchronize remote memory
>   multi-process: create IOHUB object to handle irq
>   multi-process: Retrieve PCI info from remote process
> 
> John G Johnson (1):
>   multi-process: add the concept description to
>     docs/devel/qemu-multiprocess
> 
>  docs/devel/index.rst                      |   1 +
>  docs/devel/multi-process.rst              | 966 ++++++++++++++++++++++
>  docs/multi-process.rst                    |  64 ++
>  configure                                 |  10 +
>  meson.build                               |   3 +
>  hw/remote/trace.h                         |   1 +
>  include/exec/memory.h                     |   2 +
>  include/exec/ram_addr.h                   |   2 +-
>  include/hw/pci-host/remote.h              |  31 +
>  include/hw/pci/pci_ids.h                  |   3 +
>  include/hw/remote/iohub.h                 |  42 +
>  include/hw/remote/machine.h               |  40 +
>  include/hw/remote/memory.h                |  19 +
>  include/hw/remote/mpqemu-link.h           |  99 +++
>  include/hw/remote/proxy-memory-listener.h |  28 +
>  include/hw/remote/proxy.h                 |  52 ++
>  include/io/channel.h                      |  50 ++
>  include/qemu/mmap-alloc.h                 |   4 +-
>  include/qemu/sockets.h                    |   1 +
>  include/sysemu/iothread.h                 |   6 +
>  backends/hostmem-memfd.c                  |   2 +-
>  hw/misc/ivshmem.c                         |   3 +-
>  hw/pci-host/remote.c                      |  75 ++
>  hw/remote/iohub.c                         | 119 +++
>  hw/remote/machine.c                       |  80 ++
>  hw/remote/memory.c                        |  65 ++
>  hw/remote/message.c                       | 230 ++++++
>  hw/remote/mpqemu-link.c                   | 267 ++++++
>  hw/remote/proxy-memory-listener.c         | 227 +++++
>  hw/remote/proxy.c                         | 371 +++++++++
>  hw/remote/remote-obj.c                    | 194 +++++
>  io/channel.c                              |  88 +-
>  iothread.c                                |   6 +
>  softmmu/memory.c                          |   3 +-
>  softmmu/physmem.c                         |  11 +-
>  stubs/monitor.c                           |   2 +-
>  tests/test-util-sockets.c                 |   2 +-
>  util/mmap-alloc.c                         |   7 +-
>  util/oslib-posix.c                        |   2 +-
>  util/qemu-sockets.c                       |  18 +-
>  Kconfig.host                              |   4 +
>  MAINTAINERS                               |  24 +
>  hw/Kconfig                                |   1 +
>  hw/meson.build                            |   1 +
>  hw/pci-host/Kconfig                       |   3 +
>  hw/pci-host/meson.build                   |   1 +
>  hw/remote/Kconfig                         |   4 +
>  hw/remote/meson.build                     |  13 +
>  hw/remote/trace-events                    |   4 +
>  49 files changed, 3202 insertions(+), 49 deletions(-)
>  create mode 100644 docs/devel/multi-process.rst
>  create mode 100644 docs/multi-process.rst
>  create mode 100644 hw/remote/trace.h
>  create mode 100644 include/hw/pci-host/remote.h
>  create mode 100644 include/hw/remote/iohub.h
>  create mode 100644 include/hw/remote/machine.h
>  create mode 100644 include/hw/remote/memory.h
>  create mode 100644 include/hw/remote/mpqemu-link.h
>  create mode 100644 include/hw/remote/proxy-memory-listener.h
>  create mode 100644 include/hw/remote/proxy.h
>  create mode 100644 hw/pci-host/remote.c
>  create mode 100644 hw/remote/iohub.c
>  create mode 100644 hw/remote/machine.c
>  create mode 100644 hw/remote/memory.c
>  create mode 100644 hw/remote/message.c
>  create mode 100644 hw/remote/mpqemu-link.c
>  create mode 100644 hw/remote/proxy-memory-listener.c
>  create mode 100644 hw/remote/proxy.c
>  create mode 100644 hw/remote/remote-obj.c
>  create mode 100644 hw/remote/Kconfig
>  create mode 100644 hw/remote/meson.build
>  create mode 100644 hw/remote/trace-events
> 
> -- 
> 2.25.GIT
>