[PATCH 00/41] arm: Implement GICv4

[Peter Maydell]

This patchset implements emulation of GICv4 in our TCG GIC and ITS
models, and makes the virt board use it where appropriate.

The GICv4 provides a single new feature: direct injection of virtual
interrupts from the ITS to a VM. In QEMU terms this means that if you
have an outer QEMU which is emulating a CPU with EL2, and the outer
guest passes through a PCI device (probably one emulated by the outer
QEMU) to an inner guest, interrupts from that device can go directly
to the inner guest, rather than having to go to the outer guest and
the outer guest then synthesizing virtual interrupts to the inner
guest. (If you aren't configuring the inner guest with a passthrough
PCI device then this new feature is of no interest.)

The basic structure of the patchset is as follows:

(1) There are a handful of preliminary patches fixing some minor
existing nits.

(2) The v4 ITS has some new in-guest-memory data structures and new
ITS commands that let the guest set them up. The next sequence of
patches implement all those commands. Where the command needs to
actually do something (eg "deliver a vLPI"), these patches call
functions in the redistributor which are left as unimplemented stubs
to be filled in in subsequent patches. This first chunk of patches
sticks to the data-structure handling and all the command argument
unpacking and error checking.

(3) The redistributor has a new redistributor frame (ie the amount of
guest memory used by redistributor registers is larger) with a two new
registers in it. We implement these initially as reads-as-written.

(4) The CPU interface needs relatively minor changes: as well as
looking at the list registers to determine the highest priority
pending virtual interrupt, we must also look at the highest priority
pending vLPI. We implement these changes, again leaving the interfaces
from this code into the redistributor as stubs for the moment.

(5) Now we can fill in all the stub code in the redistributor.  This
is almost all working with the pending and config tables for virtual
LPIs. (Side note: in real hardware some of this work is done by the
ITS rather than the redistributor, but in our implementation we split
between the two source files slightly differently. I've made the vLPI
handling follow the pattern of the existing LPI handling.)

(6) Finally, we can update the ID registers which tell the guest about
the presence of v4 features, allow the GIC device to accept 4 as a
value for its QOM revision property, and make the virt board set that
when appropriate.

General notes:

Since the only useful thing in GICv4 is direct virtual interrupt
injection, it isn't expected that you would have a system with a GICv4
and a CPU without EL2. So I've made this an error, and the virt board
will only use GICv4 if the user also enables emulation of
virtualization.

Because the redistributor frame is twice the size in GICv4, the
number of redistributors we can fit into a given area of memory
is reduced. This means that when using GICv4 the maximum number
of CPUs supported on the virt board drops from 512 to 317. (No,
I'm not sure why this is 317 and not 256 :-))

I have not particularly considered performance in this initial
implementation. In particular, we will do a complete re-scan of a
virtual LPI pending table every time the outer guest reschedules a
vCPU (and writes GICR_VPENDBASER). The spec provides scope for
optimisation here, by allowing part of the LPI table to have IMPDEF
contents, which we could in principle use to cache information like
the current highest priority pending vLPI. Given that emulating
nested guests with PCI passthrough is a fairly niche activity,
I propose that we not do this unless the three people doing that
complain about this all being too slow :-)

Tested with a Linux kernel passing through a virtio-blk device
to an inner Linux VM with KVM/QEMU. (NB that to get the outer
Linux kernel to actually use the new GICv4 functionality you
need to pass it "kvm-arm.vgic_v4_enable=1", as the kernel
will not use it by default.)

thanks
-- PMM

Peter Maydell (41):
  hw/intc/arm_gicv3_its: Add missing blank line
  hw/intc/arm_gicv3: Sanity-check num-cpu property
  hw/intc/arm_gicv3: Insist that redist region capacity matches CPU count
  hw/intc/arm_gicv3: Report correct PIDR0 values for ID registers
  target/arm/cpu.c: ignore VIRQ and VFIQ if no EL2
  hw/intc/arm_gicv3_its: Factor out "is intid a valid LPI ID?"
  hw/intc/arm_gicv3_its: Implement GITS_BASER2 for GICv4
  hw/intc/arm_gicv3_its: Implement VMAPI and VMAPTI
  hw/intc/arm_gicv3_its: Implement VMAPP
  hw/intc/arm_gicv3_its: Distinguish success and error cases of CMD_CONTINUE
  hw/intc/arm_gicv3_its: Factor out "find ITE given devid, eventid"
  hw/intc/arm_gicv3_its: Factor out CTE lookup sequence
  hw/intc/arm_gicv3_its: Split out process_its_cmd() physical interrupt code
  hw/intc/arm_gicv3_its: Handle virtual interrupts in process_its_cmd()
  hw/intc/arm_gicv3: Keep pointers to every connected ITS
  hw/intc/arm_gicv3_its: Implement VMOVP
  hw/intc/arm_gicv3_its: Implement VSYNC
  hw/intc/arm_gicv3_its: Implement INV command properly
  hw/intc/arm_gicv3_its: Implement INV for virtual interrupts
  hw/intc/arm_gicv3_its: Implement VMOVI
  hw/intc/arm_gicv3_its: Implement VINVALL
  hw/intc/arm_gicv3: Implement GICv4's new redistributor frame
  hw/intc/arm_gicv3: Implement new GICv4 redistributor registers
  hw/intc/arm_gicv3_cpuif: Split "update vIRQ/vFIQ" from
    gicv3_cpuif_virt_update()
  hw/intc/arm_gicv3_cpuif: Support vLPIs
  hw/intc/arm_gicv3_cpuif: Don't recalculate maintenance irq unnecessarily
  hw/intc/arm_gicv3_redist: Factor out "update hpplpi for one LPI" logic
  hw/intc/arm_gicv3_redist: Factor out "update hpplpi for all LPIs" logic
  hw/intc/arm_gicv3_redist: Recalculate hppvlpi on VPENDBASER writes
  hw/intc/arm_gicv3_redist: Factor out "update bit in pending table" code
  hw/intc/arm_gicv3_redist: Implement gicv3_redist_process_vlpi()
  hw/intc/arm_gicv3_redist: Implement gicv3_redist_vlpi_pending()
  hw/intc/arm_gicv3_redist: Use set_pending_table_bit() in mov handling
  hw/intc/arm_gicv3_redist: Implement gicv3_redist_mov_vlpi()
  hw/intc/arm_gicv3_redist: Implement gicv3_redist_vinvall()
  hw/intc/arm_gicv3_redist: Implement gicv3_redist_inv_vlpi()
  hw/intc/arm_gicv3: Update ID and feature registers for GICv4
  hw/intc/arm_gicv3: Allow 'revision' property to be set to 4
  hw/arm/virt: Use VIRT_GIC_VERSION_* enum values in create_gic()
  hw/arm/virt: Abstract out calculation of redistributor region capacity
  hw/arm/virt: Support TCG GICv4

 docs/system/arm/virt.rst               |   5 +-
 hw/intc/gicv3_internal.h               | 231 ++++++-
 include/hw/arm/virt.h                  |  19 +-
 include/hw/intc/arm_gicv3_common.h     |  13 +
 include/hw/intc/arm_gicv3_its_common.h |   1 +
 hw/arm/virt.c                          | 102 ++-
 hw/intc/arm_gicv3_common.c             |  54 +-
 hw/intc/arm_gicv3_cpuif.c              | 195 +++++-
 hw/intc/arm_gicv3_dist.c               |   7 +-
 hw/intc/arm_gicv3_its.c                | 876 ++++++++++++++++++++-----
 hw/intc/arm_gicv3_its_kvm.c            |   2 +
 hw/intc/arm_gicv3_kvm.c                |   5 +
 hw/intc/arm_gicv3_redist.c             | 480 +++++++++++---
 target/arm/cpu.c                       |  12 +-
 hw/intc/trace-events                   |  18 +-
 15 files changed, 1695 insertions(+), 325 deletions(-)

-- 
2.25.1