[Qemu-devel] [RFC v2 00/11] Base enabling patches for MTTCG

[Alex Bennée]

Hi,

This is the second iteration of the RFC patch set aims to provide the
basic framework for MTTCG. There wasn't much feedback on the first
posting but there has been some changes:

  - fixed bugs in single-thread mode w.r.t shutdown
  - tested multi-thread mode against (non)locking-test
  - included docs/multi-thread-tcg.txt design document
  - clean-ups to attribution and s-o-b tags
  - dropped tcg: move tb_find_fast outside the tb_lock critical
    - this was breaking the pxe tests in make check
  - pulled in tcg: move tb_invalidated_flag to CPUState (from Serge)
  - dropped cpu-exec: elide more icount code if CONFIG_USER_ONLY (to Serge)

The biggest changes were to the last patch that enables multi-threading
to fix various bugs. The single-thread case was failing to shutdown
due not bringing the system to wait on the halt_cond once no vCPUs
could be scheduled anymore. Conversely the multi-threaded mode was
crashing on the power-up of secondary vCPUs as the reset of vCPUs at
the start would leave the thread spinning instead of waiting on the
halt_cond until PSCI has cleanly enabled the vCPU.

The branch can be found at:

  https://github.com/stsquad/qemu/tree/mttcg/base-patches-v2

It has been built up on top of:

  https://github.com/stsquad/qemu/tree/mttcg/tb-and-tcg-cleanups

Which is just a collection of Serge's TCG cleanups not directly tied
to MTTCG which will hopefully get merged in due course well before
these patches.

What's next
===========

This series should now behave as normal in the default single threaded
mode - any regressions are bugs that I'd like to know about.

multi-threaded mode works for any guests that don't make assumptions
about atomicity, memory consistency or cache flushes. Basically simple
kvm-unit-test cases. I've been using the simple locking-test:

  ./arm/run ./arm/locking-test.flat -smp 4 [-tcg mttcg=on]

Default:
real    0m42.335s
user    0m42.296s
sys     0m0.024s

Multi-threaded:
real    1m32.928s
user    2m15.704s
sys     3m4.840s

Obviously the performance in this simple test case is a regression
(mainly due to lock contention). This is to be expected and something
I hope can be addressed as this series is built on with ways of
reducing lock contention and safely handling the various invalidation
operations.

I hope this works as a good base for further MTTCG development. My
next step will be to re-build the remaining patches from Fred's
multi_tcg_v8 so the changes to enable ARMv7 on x86 can be reviewed.
This basically involves adding:

  - async_safe_work and associated flushes
  - atomic safety
  - defaulting to on

Obviously I await the various trees from Serge, Alvise and Emilio with interest 
;-)

In the meantime any review comments gratefully received.

Cheers,

Alex


Alex Bennée (5):
  cpus: make all_vcpus_paused() return bool
  docs: new design document multi-thread-tcg.txt (DRAFTING)
  target-arm/psci.c: wake up sleeping CPUs
  tcg: cpus rm tcg_exec_all()
  tcg: add kick timer for single-threaded vCPU emulation

Jan Kiszka (1):
  tcg: drop global lock during TCG code execution

KONRAD Frederic (3):
  tcg: protect TBContext with tb_lock.
  tcg: add options for enabling MTTCG
  tcg: enable thread-per-vCPU

Paolo Bonzini (2):
  tcg: move tb_invalidated_flag to CPUState
  tcg: comment on which functions have to be called with tb_lock held

 cpu-exec-common.c         |   1 -
 cpu-exec.c                |  46 +++---
 cpus.c                    | 357 +++++++++++++++++++++++++++++++---------------
 cputlb.c                  |   1 +
 docs/multi-thread-tcg.txt | 184 ++++++++++++++++++++++++
 exec.c                    |  16 +++
 hw/i386/kvmvapic.c        |   6 +
 include/exec/exec-all.h   |   7 +-
 include/qom/cpu.h         |  19 +++
 include/sysemu/cpus.h     |   2 +
 memory.c                  |   2 +
 qemu-options.hx           |  14 ++
 softmmu_template.h        |  17 +++
 target-arm/psci.c         |   2 +
 target-i386/smm_helper.c  |   7 +
 tcg/tcg.h                 |   2 +
 translate-all.c           |  89 ++++++++----
 vl.c                      |  12 +-
 18 files changed, 613 insertions(+), 171 deletions(-)
 create mode 100644 docs/multi-thread-tcg.txt

-- 
2.7.4