[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v6 36/36] multi-process: add configure and usage information
From: |
elena . ufimtseva |
Subject: |
[PATCH v6 36/36] multi-process: add configure and usage information |
Date: |
Mon, 6 Apr 2020 02:41:26 -0700 |
From: Elena Ufimtseva <address@hidden>
Signed-off-by: Elena Ufimtseva <address@hidden>
Signed-off-by: Jagannathan Raman <address@hidden>
Signed-off-by: John G Johnson <address@hidden>
---
docs/multi-process.rst | 85 +++++++++++++++++++++++++
scripts/mpqemu-launcher-perf-mode.py | 93 ++++++++++++++++++++++++++++
scripts/mpqemu-launcher.py | 54 ++++++++++++++++
3 files changed, 232 insertions(+)
create mode 100644 docs/multi-process.rst
create mode 100755 scripts/mpqemu-launcher-perf-mode.py
create mode 100755 scripts/mpqemu-launcher.py
diff --git a/docs/multi-process.rst b/docs/multi-process.rst
new file mode 100644
index 0000000000..9d53667b4e
--- /dev/null
+++ b/docs/multi-process.rst
@@ -0,0 +1,85 @@
+Multi-process QEMU
+==================
+
+This document describes how to configure and use multi-process qemu.
+For the design document refer to docs/devel/qemu-multiprocess.
+
+1) Configuration
+----------------
+
+To enable support for multi-process add --enable-mpqemu
+to the list of options for the "configure" script.
+
+
+2) Usage
+--------
+
+Multi-process QEMU requires an orchestrator to launch. Please refer to a
+light-weight python based orchestrator for mpqemu in
+scripts/mpqemu-launcher.py to lauch QEMU in multi-process mode.
+
+scripts/mpqemu-launcher-perf-mode.py launches in "perf" mode. In this mode,
+the same QEMU process connects to multiple remote devices, each emulated in
+a separate process.
+
+As of now, we only support the emulation of lsi53c895a in a separate process.
+
+Following is a description of command-line used to launch mpqemu.
+
+* Orchestrator:
+
+ - The Orchestrator creates a unix socketpair
+
+ - It launches the remote process and passes one of the
+ sockets to it via command-line.
+
+ - It then launches QEMU and specifies the other socket as an option
+ to the Proxy device object
+
+* Remote Process:
+
+ - The first command-line option in the remote process is one of the
+ sockets created by the Orchestrator
+
+ - The remaining options are no different from how one launches QEMU with
+ devices. The only other requirement is each PCI device must have a
+ unique ID specified to it. This is needed to pair remote device with the
+ Proxy object.
+
+ - Example command-line for the remote process is as follows:
+
+ /usr/bin/qemu-scsu-dev 4 \
+ -device lsi53c895a,id=lsi0 \
+ -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2 \
+ -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0
+
+* QEMU:
+
+ - Since parts of the RAM are shared between QEMU & remote process, a
+ memory-backend-file is required to facilitate this, as follows:
+
+ -object memory-backend-file,id=mem,mem-path=/dev/shm/,size=4096M,share=on
+
+ - A "pci-proxy-dev" device is created for each of the PCI devices emulated
+ in the remote process. A "socket" sub-option specifies the other end of
+ unix channel created by orchestrator. The "id" sub-option must be specified
+ and should be the same as the "id" specified for the remote PCI device
+
+ - Example commandline for QEMU is as follows:
+
+ -device pci-proxy-dev,id=lsi0,socket=3
+
+* Monitor / QMP:
+
+ - The remote process supports QEMU monitor. It could be specified using the
+ "-monitor" or "-qmp" command-line options
+
+ - As an example, one could connect to the monitor by adding the following
+ to the command-line of the remote process
+
+ -monitor unix:/home/qmp-sock,server,nowait
+
+ - The user could connect to the monitor using the qmp script or using
+ "socat" as outlined below:
+
+ socat /home/qmp-sock stdio
diff --git a/scripts/mpqemu-launcher-perf-mode.py
b/scripts/mpqemu-launcher-perf-mode.py
new file mode 100755
index 0000000000..a9be82faf5
--- /dev/null
+++ b/scripts/mpqemu-launcher-perf-mode.py
@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+
+import socket
+import os
+import subprocess
+import time
+
+PROC_QEMU='/usr/bin/qemu-system-x86_64'
+
+PROC_REMOTE='/usr/bin/qemu-scsi-dev'
+
+proxy_1, remote_1 = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+proxy_2, remote_2 = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+proxy_3, remote_3 = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+
+remote_cmd_1 = [ PROC_REMOTE,
\
+ str(remote_1.fileno()),
\
+ '-device', 'lsi53c895a,id=lsi1',
\
+ '-drive', 'id=drive_image1,'
\
+ 'file=/build/ol7-nvme-test-1.qcow2',
\
+ '-device', 'scsi-hd,id=drive1,drive=drive_image1,'
\
+ 'bus=lsi1.0,scsi-id=0',
\
+ ]
+
+remote_cmd_2 = [ PROC_REMOTE,
\
+ str(remote_2.fileno()),
\
+ '-device', 'lsi53c895a,id=lsi2',
\
+ '-drive', 'id=drive_image2,'
\
+ 'file=/build/ol7-nvme-test-2.qcow2',
\
+ '-device', 'scsi-hd,id=drive2,drive=drive_image2,'
\
+ 'bus=lsi2.0,scsi-id=0'
\
+ ]
+
+remote_cmd_3 = [ PROC_REMOTE,
\
+ str(remote_3.fileno()),
\
+ '-device', 'lsi53c895a,id=lsi3',
\
+ '-drive', 'id=drive_image3,'
\
+ 'file=/build/ol7-nvme-test-3.qcow2',
\
+ '-device', 'scsi-hd,id=drive3,drive=drive_image3,'
\
+ 'bus=lsi3.0,scsi-id=0'
\
+ ]
+
+proxy_cmd = [ PROC_QEMU,
\
+ '-name', 'OL7.4',
\
+ '-machine', 'q35,accel=kvm',
\
+ '-smp', 'sockets=1,cores=1,threads=1',
\
+ '-m', '2048',
\
+ '-object', 'memory-backend-file,id=sysmem-file,'
\
+ 'mem-path=/dev/shm/test-mem,size=2G,share=on',
\
+ '-numa', 'node,memdev=sysmem-file',
\
+ '-device', 'virtio-scsi-pci,id=virtio_scsi_pci0',
\
+ '-drive', 'id=drive_image1,if=none,format=qcow2,'
\
+ 'file=/home/ol7-hdd-1.qcow2',
\
+ '-device', 'scsi-hd,id=image1,drive=drive_image1,'
\
+ 'bus=virtio_scsi_pci0.0',
\
+ '-boot', 'd',
\
+ '-vnc', ':0',
\
+ '-device', 'pci-proxy-dev,id=lsi1,'
\
+ 'socket='+str(proxy_1.fileno()),
\
+ '-device', 'pci-proxy-dev,id=lsi2,'
\
+ 'socket='+str(proxy_2.fileno()),
\
+ '-device', 'pci-proxy-dev,id=lsi3,'
\
+ 'socket='+str(proxy_3.fileno())
\
+ ]
+
+
+pid = os.fork();
+if pid == 0:
+ # In remote_1
+ print('Launching Remote process 1');
+ process = subprocess.Popen(remote_cmd_1, pass_fds=[remote_1.fileno()])
+ os._exit(0)
+
+
+pid = os.fork();
+if pid == 0:
+ # In remote_2
+ print('Launching Remote process 2');
+ process = subprocess.Popen(remote_cmd_2, pass_fds=[remote_2.fileno()])
+ os._exit(0)
+
+
+pid = os.fork();
+if pid == 0:
+ # In remote_3
+ print('Launching Remote process 3');
+ process = subprocess.Popen(remote_cmd_3, pass_fds=[remote_3.fileno()])
+ os._exit(0)
+
+
+print('Launching Proxy process');
+process = subprocess.Popen(proxy_cmd, pass_fds=[proxy_1.fileno(),
\
+ proxy_2.fileno(), proxy_3.fileno()])
diff --git a/scripts/mpqemu-launcher.py b/scripts/mpqemu-launcher.py
new file mode 100755
index 0000000000..28d6aa441f
--- /dev/null
+++ b/scripts/mpqemu-launcher.py
@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+import socket
+import os
+import subprocess
+import time
+
+PROC_QEMU='/usr/bin/qemu-system-x86_64'
+
+PROC_REMOTE='/usr/bin/qemu-scsi-dev'
+
+proxy, remote = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+
+remote_cmd = [ PROC_REMOTE,
\
+ str(remote.fileno()),
\
+ '-device', 'lsi53c895a,id=lsi1',
\
+ '-drive', 'id=drive_image1,file=/build/ol7-nvme-test-1.qcow2',
\
+ '-device', 'scsi-hd,id=drive1,drive=drive_image1,bus=lsi1.0,'
\
+ 'scsi-id=0',
\
+ '-device', 'lsi53c895a,id=lsi2',
\
+ '-drive', 'id=drive_image2,file=/build/ol7-nvme-test-2.qcow2',
\
+ '-device', 'scsi-hd,id=drive2,drive=drive_image2,bus=lsi2.0,'
\
+ 'scsi-id=0'
\
+ ]
+
+proxy_cmd = [ PROC_QEMU,
\
+ '-name', 'OL7.4',
\
+ '-machine', 'q35,accel=kvm',
\
+ '-smp', 'sockets=1,cores=1,threads=1',
\
+ '-m', '2048',
\
+ '-object', 'memory-backend-file,id=sysmem-file,'
\
+ 'mem-path=/dev/shm/test-mem,size=2G,share=on',
\
+ '-numa', 'node,memdev=sysmem-file',
\
+ '-device', 'virtio-scsi-pci,id=virtio_scsi_pci0',
\
+ '-drive', 'id=drive_image1,if=none,format=qcow2,'
\
+ 'file=/home/ol7-hdd-1.qcow2',
\
+ '-device', 'scsi-hd,id=image1,drive=drive_image1,'
\
+ 'bus=virtio_scsi_pci0.0',
\
+ '-boot', 'd',
\
+ '-vnc', ':0',
\
+ '-device', 'pci-proxy-dev,id=lsi1,socket='+str(proxy.fileno()),
\
+ '-device', 'pci-proxy-dev,id=lsi2,socket='+str(proxy.fileno())
\
+ ]
+
+
+pid = os.fork();
+
+if pid:
+ # In Proxy
+ print('Launching QEMU with Proxy object');
+ process = subprocess.Popen(proxy_cmd, pass_fds=[proxy.fileno()])
+else:
+ # In remote
+ print('Launching Remote process');
+ process = subprocess.Popen(remote_cmd, pass_fds=[remote.fileno()])
--
2.25.GIT
- [PATCH v6 19/36] multi-process: Connect Proxy Object with device in the remote process, (continued)
- [PATCH v6 19/36] multi-process: Connect Proxy Object with device in the remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 22/36] multi-process: Synchronize remote memory, elena . ufimtseva, 2020/04/06
- [PATCH v6 26/36] multi-process: add parse_cmdline in remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 20/36] multi-process: Forward PCI config space acceses to the remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 25/36] multi-process: Introduce build flags to separate remote process code, elena . ufimtseva, 2020/04/06
- [PATCH v6 28/36] multi-process: send heartbeat messages to remote, elena . ufimtseva, 2020/04/06
- [PATCH v6 29/36] multi-process: handle heartbeat messages in remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 31/36] multi-process/mon: choose HMP commands based on target, elena . ufimtseva, 2020/04/06
- [PATCH v6 23/36] multi-process: create IOHUB object to handle irq, elena . ufimtseva, 2020/04/06
- [PATCH v6 33/36] multi-process/mon: enable QMP module support in the remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 36/36] multi-process: add configure and usage information,
elena . ufimtseva <=
- [PATCH v6 32/36] multi-process/mon: stub functions to enable QMP module for remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 35/36] multi-process: add the concept description to docs/devel/qemu-multiprocess, elena . ufimtseva, 2020/04/06
- [PATCH v6 02/36] multi-process: Refactor machine_init and exit notifiers, elena . ufimtseva, 2020/04/06
- [PATCH v6 03/36] command-line: refractor parser code, elena . ufimtseva, 2020/04/06
- [PATCH v6 06/36] monitor: destaticize HMP commands, elena . ufimtseva, 2020/04/06
- [PATCH v6 09/36] multi-process: Add config option for multi-process QEMU, elena . ufimtseva, 2020/04/06
- [PATCH v6 11/36] multi-process: define mpqemu-link object, elena . ufimtseva, 2020/04/06
- [PATCH v6 24/36] multi-process: Retrieve PCI info from remote process, elena . ufimtseva, 2020/04/06
- [PATCH v6 27/36] multi-process: add support to parse device option, elena . ufimtseva, 2020/04/06