Re: [PATCH v20 00/20] Initial support for multi-process Qemu

[Jag Raman]

> On Jan 21, 2021, at 12:07 PM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> 
> On Tue, Jan 19, 2021 at 03:28:17PM -0500, Jagannathan Raman wrote:
>> Hi,
>> 
>> This is the v20 of the patchset. This version has the following changes:
>> 
>> [PATCH v18 08/20] io: add qio_channel_readv_full_all_eof &
>>               qio_channel_readv_full_all helpers
>> - Considers fds as part of the "data" for both
>>   qio_channel_readv_full_all_eof() &
>>   qio_channel_readv_full_all().
>> - Addresses some of the error scenarios
>>   which were not addressed in the previous revisions
>> 
>> To touch upon the history of this project, we posted the Proof Of Concept
>> patches before the BoF session in 2018. Subsequently, we have posted 19
>> versions on the qemu-devel mailing list. You can find them by following
>> the links below ([1] - [19]). Following people contributed to the design and
>> implementation of this project:
>> Jagannathan Raman <jag.raman@oracle.com>
>> Elena Ufimtseva <elena.ufimtseva@oracle.com>
>> John G Johnson <john.g.johnson@oracle.com>
>> Stefan Hajnoczi <stefanha@redhat.com>
>> Konrad Wilk <konrad.wilk@oracle.com>
>> Kanth Ghatraju <kanth.ghatraju@oracle.com>
>> 
>> We would like to thank you for your feedback in the
>> design and implementation of this project. Qemu wiki page:
>> https://wiki.qemu.org/Features/MultiProcessQEMU
>> 
>> For the full concept writeup about QEMU multi-process, please
>> refer to docs/devel/qemu-multiprocess.rst. Also, see
>> docs/qemu-multiprocess.txt for usage information.
>> 
>> Thank you for reviewing this series!
>> 
>> [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
>> [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
>> [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
>> [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
>> [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
>> [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html
>> [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html
>> [7]: 
>> https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/
>> [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html
>> [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html
>> [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html
>> [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html
>> [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/
>> [13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html
>> [14]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg768376.html
>> [15]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg769178.html
>> [16]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg771455.html
>> [17]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772512.html
>> [18]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772602.html
>> [19]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772980.html
>> 
>> Elena Ufimtseva (8):
>> multi-process: add configure and usage information
>> io: add qio_channel_writev_full_all helper
>> io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all
>>   helpers
>> multi-process: define MPQemuMsg format and transmission functions
>> multi-process: introduce proxy object
>> multi-process: add proxy communication functions
>> multi-process: Forward PCI config space acceses to the remote process
>> multi-process: perform device reset in the remote process
>> 
>> Jagannathan Raman (11):
>> memory: alloc RAM from file at offset
>> multi-process: Add config option for multi-process QEMU
>> multi-process: setup PCI host bridge for remote device
>> multi-process: setup a machine object for remote device process
>> multi-process: Initialize message handler in remote device
>> multi-process: Associate fd of a PCIDevice with its object
>> multi-process: setup memory manager for remote device
>> multi-process: PCI BAR read/write handling for proxy & remote
>>   endpoints
>> multi-process: Synchronize remote memory
>> multi-process: create IOHUB object to handle irq
>> multi-process: Retrieve PCI info from remote process
>> 
>> John G Johnson (1):
>> multi-process: add the concept description to
>>   docs/devel/qemu-multiprocess
>> 
>> docs/devel/index.rst                      |   1 +
>> docs/devel/multi-process.rst              | 966 
>> ++++++++++++++++++++++++++++++
>> docs/multi-process.rst                    |  64 ++
>> configure                                 |  10 +
>> meson.build                               |   5 +-
>> hw/remote/trace.h                         |   1 +
>> include/exec/memory.h                     |   2 +
>> include/exec/ram_addr.h                   |   2 +-
>> include/hw/pci-host/remote.h              |  30 +
>> include/hw/pci/pci_ids.h                  |   3 +
>> include/hw/remote/iohub.h                 |  42 ++
>> include/hw/remote/machine.h               |  38 ++
>> include/hw/remote/memory.h                |  19 +
>> include/hw/remote/mpqemu-link.h           |  99 +++
>> include/hw/remote/proxy-memory-listener.h |  28 +
>> include/hw/remote/proxy.h                 |  48 ++
>> include/io/channel.h                      |  78 +++
>> include/qemu/mmap-alloc.h                 |   4 +-
>> include/sysemu/iothread.h                 |   6 +
>> backends/hostmem-memfd.c                  |   2 +-
>> hw/misc/ivshmem.c                         |   3 +-
>> hw/pci-host/remote.c                      |  75 +++
>> hw/remote/iohub.c                         | 119 ++++
>> hw/remote/machine.c                       |  80 +++
>> hw/remote/memory.c                        |  65 ++
>> hw/remote/message.c                       | 230 +++++++
>> hw/remote/mpqemu-link.c                   | 267 +++++++++
>> hw/remote/proxy-memory-listener.c         | 227 +++++++
>> hw/remote/proxy.c                         | 379 ++++++++++++
>> hw/remote/remote-obj.c                    | 203 +++++++
>> io/channel.c                              | 121 +++-
>> iothread.c                                |   6 +
>> softmmu/memory.c                          |   3 +-
>> softmmu/physmem.c                         |  11 +-
>> util/mmap-alloc.c                         |   7 +-
>> util/oslib-posix.c                        |   2 +-
>> Kconfig.host                              |   4 +
>> MAINTAINERS                               |  24 +
>> hw/Kconfig                                |   1 +
>> hw/meson.build                            |   1 +
>> hw/pci-host/Kconfig                       |   3 +
>> hw/pci-host/meson.build                   |   1 +
>> hw/remote/Kconfig                         |   4 +
>> hw/remote/meson.build                     |  13 +
>> hw/remote/trace-events                    |   4 +
>> 45 files changed, 3265 insertions(+), 36 deletions(-)
>> create mode 100644 docs/devel/multi-process.rst
>> create mode 100644 docs/multi-process.rst
>> create mode 100644 hw/remote/trace.h
>> create mode 100644 include/hw/pci-host/remote.h
>> create mode 100644 include/hw/remote/iohub.h
>> create mode 100644 include/hw/remote/machine.h
>> create mode 100644 include/hw/remote/memory.h
>> create mode 100644 include/hw/remote/mpqemu-link.h
>> create mode 100644 include/hw/remote/proxy-memory-listener.h
>> create mode 100644 include/hw/remote/proxy.h
>> create mode 100644 hw/pci-host/remote.c
>> create mode 100644 hw/remote/iohub.c
>> create mode 100644 hw/remote/machine.c
>> create mode 100644 hw/remote/memory.c
>> create mode 100644 hw/remote/message.c
>> create mode 100644 hw/remote/mpqemu-link.c
>> create mode 100644 hw/remote/proxy-memory-listener.c
>> create mode 100644 hw/remote/proxy.c
>> create mode 100644 hw/remote/remote-obj.c
>> create mode 100644 hw/remote/Kconfig
>> create mode 100644 hw/remote/meson.build
>> create mode 100644 hw/remote/trace-events
>> 
>> -- 
>> 1.8.3.1
>> 
> 
> I will merge this when Daniel Berrange has reviewed it.
> 
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Thank you for confirming, Stefan!