[PATCH v2 00/17] Switch iotests to using Async QMP

[John Snow]

Based-on: <20210915162955.333025-1-jsnow@redhat.com>
          [PATCH v4 00/27] python: introduce Asynchronous QMP package
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-aqmp-iotest-wrapper
CI: https://gitlab.com/jsnow/qemu/-/pipelines/375637927

Hiya,

This series continues where the first AQMP series left off and adds a
synchronous 'legacy' wrapper around the new AQMP interface, then drops
it straight into iotests to prove that AQMP is functional and totally
cool and fine. The disruption and churn to iotests is extremely minimal.
(There's actually a net negative SLOC in tests/qemu-iotests.)

In the event that a regression happens and I am not physically proximate
to inflict damage upon, one may set the QEMU_PYTHON_LEGACY_QMP variable
to any non-empty string as it pleases you to engage the QMP machinery
you are used to.

I'd like to try and get this committed early in the 6.2 development
cycle to give ample time to smooth over any possible regressions. I've
tested it locally and via gitlab CI, across Python versions 3.6 through
3.10, and "worksforme". If something bad happens, we can revert the
actual switch-flip very trivially.

Layout:

Patches 1-7: ./python/qemu/aqmp changes that serve as pre-requisites.
Patches 8-12: other ./python changes that ease the transition.
Patches 13-14: iotest changes to support the new QMP backend.
Patches 15-17: Make the switch.

V2:

001/17:[----] [--] 'python/aqmp: add greeting property to QMPClient'
002/17:[----] [--] 'python/aqmp: add .empty() method to EventListener'
003/17:[----] [--] 'python/aqmp: Return cleared events from 
EventListener.clear()'
004/17:[0007] [FC] 'python/aqmp: add send_fd_scm'
005/17:[down] 'python/aqmp: Add dict conversion method to Greeting object'
006/17:[down] 'python/aqmp: Reduce severity of EOFError-caused loop 
terminations'
007/17:[down] 'python/aqmp: Disable logging messages by default'

008/17:[0002] [FC] 'python/qmp: clear events on get_events() call'
009/17:[----] [--] 'python/qmp: add send_fd_scm directly to QEMUMonitorProtocol'
010/17:[----] [--] 'python, iotests: remove socket_scm_helper'
011/17:[0013] [FC] 'python/machine: remove has_quit argument'
012/17:[down] 'python/machine: Handle QMP errors on close more meticulously'

013/17:[0009] [FC] 'iotests: Accommodate async QMP Exception classes'
014/17:[down] 'iotests: Conditionally silence certain AQMP errors'

015/17:[0016] [FC] 'python/aqmp: Create sync QMP wrapper for iotests'
016/17:[0002] [FC] 'python/aqmp: Remove scary message'
017/17:[----] [--] 'python, iotests: replace qmp with aqmp'

- Rebased on jsnow/python, which was recently rebased on origin/master.
- Make aqmp's send_fd_scm method bark if the socket isn't AF_UNIX (Hanna)
- Uh... modify send_fd_scm so it doesn't break when Python 3.11 comes out ...
  See the commit message for more detail.
- Drop the "python/aqmp: Create MessageModel and StandaloneModel class"
  patch and replace with a far simpler method that just adds an
  _asdict() method.
- Add patches 06 and 07 to change how the AQMP library handles logging.
- Adjust docstring in patch 08 (Hanna)
- Rename "_has_quit" attribute to "_quid_issued" (Hanna)
- Renamed patch 12, simplified the logic in _soft_shutdown a tiny bit.
- Fixed bad exception handling logic in 13 (Hanna)
- Introduce a helper in patch 14 to silence log output when it's unwanted.
- Small addition of _get_greeting() helper in patch 15, coinciding with the
  new patch 05 here.
- Contextual changes in 16.

John Snow (17):
  python/aqmp: add greeting property to QMPClient
  python/aqmp: add .empty() method to EventListener
  python/aqmp: Return cleared events from EventListener.clear()
  python/aqmp: add send_fd_scm
  python/aqmp: Add dict conversion method to Greeting object
  python/aqmp: Reduce severity of EOFError-caused loop terminations
  python/aqmp: Disable logging messages by default
  python/qmp: clear events on get_events() call
  python/qmp: add send_fd_scm directly to QEMUMonitorProtocol
  python, iotests: remove socket_scm_helper
  python/machine: remove has_quit argument
  python/machine: Handle QMP errors on close more meticulously
  iotests: Accommodate async QMP Exception classes
  iotests: Conditionally silence certain AQMP errors
  python/aqmp: Create sync QMP wrapper for iotests
  python/aqmp: Remove scary message
  python, iotests: replace qmp with aqmp

 tests/qemu-iotests/socket_scm_helper.c    | 136 ----------------------
 python/qemu/aqmp/__init__.py              |  14 +--
 python/qemu/aqmp/events.py                |  15 ++-
 python/qemu/aqmp/legacy.py                | 135 +++++++++++++++++++++
 python/qemu/aqmp/models.py                |  13 +++
 python/qemu/aqmp/protocol.py              |   7 +-
 python/qemu/aqmp/qmp_client.py            |  27 +++++
 python/qemu/machine/machine.py            | 133 +++++++++++----------
 python/qemu/machine/qtest.py              |   2 -
 python/qemu/qmp/__init__.py               |  27 +++--
 python/qemu/qmp/qmp_shell.py              |   1 -
 scripts/simplebench/bench_block_job.py    |   3 +-
 tests/Makefile.include                    |   1 -
 tests/meson.build                         |   4 -
 tests/qemu-iotests/040                    |   7 +-
 tests/qemu-iotests/218                    |   2 +-
 tests/qemu-iotests/255                    |   2 +-
 tests/qemu-iotests/iotests.py             |  23 +++-
 tests/qemu-iotests/meson.build            |   5 -
 tests/qemu-iotests/testenv.py             |   8 +-
 tests/qemu-iotests/tests/mirror-top-perms |  12 +-
 21 files changed, 315 insertions(+), 262 deletions(-)
 delete mode 100644 tests/qemu-iotests/socket_scm_helper.c
 create mode 100644 python/qemu/aqmp/legacy.py
 delete mode 100644 tests/qemu-iotests/meson.build

-- 
2.31.1