Re: [PATCH for-9.2 v6 12/12] docs/specs: add riscv-iommu

[Alistair Francis]

On Fri, Aug 2, 2024 at 1:46 AM Daniel Henrique Barboza
<dbarboza@ventanamicro.com> wrote:
>
> Add a simple guideline to use the existing RISC-V IOMMU support we just
> added.
>
> This doc will be updated once we add the riscv-iommu-sys device.
>
> Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>

Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
>  docs/specs/index.rst       |  1 +
>  docs/specs/riscv-iommu.rst | 80 ++++++++++++++++++++++++++++++++++++++
>  docs/system/riscv/virt.rst | 13 +++++++
>  3 files changed, 94 insertions(+)
>  create mode 100644 docs/specs/riscv-iommu.rst
>
> diff --git a/docs/specs/index.rst b/docs/specs/index.rst
> index 1484e3e760..c68cd9ae6c 100644
> --- a/docs/specs/index.rst
> +++ b/docs/specs/index.rst
> @@ -33,3 +33,4 @@ guest hardware that is specific to QEMU.
>     virt-ctlr
>     vmcoreinfo
>     vmgenid
> +   riscv-iommu
> diff --git a/docs/specs/riscv-iommu.rst b/docs/specs/riscv-iommu.rst
> new file mode 100644
> index 0000000000..e3b8f7208e
> --- /dev/null
> +++ b/docs/specs/riscv-iommu.rst
> @@ -0,0 +1,80 @@
> +.. _riscv-iommu:
> +
> +RISC-V IOMMU support for RISC-V machines
> +========================================
> +
> +QEMU implements a RISC-V IOMMU emulation based on the RISC-V IOMMU spec
> +version 1.0 `iommu1.0`_.
> +
> +The emulation includes a PCI reference device, riscv-iommu-pci, that QEMU
> +RISC-V boards can use.  The 'virt' RISC-V machine is compatible with this
> +device.
> +
> +riscv-iommu-pci reference device
> +--------------------------------
> +
> +This device implements the RISC-V IOMMU emulation as recommended by the 
> section
> +"Integrating an IOMMU as a PCIe device" of `iommu1.0`_: a PCI device with 
> base
> +class 08h, sub-class 06h and programming interface 00h.
> +
> +As a reference device it doesn't implement anything outside of the 
> specification,
> +so it uses a generic default PCI ID given by QEMU: 1b36:0014.
> +
> +To include the device in the 'virt' machine:
> +
> +.. code-block:: bash
> +
> +  $ qemu-system-riscv64 -M virt -device riscv-iommu-pci,[optional_pci_opts] 
> (...)
> +
> +This will add a RISC-V IOMMU PCI device in the board following any additional
> +PCI parameters (like PCI bus address).  The behavior of the RISC-V IOMMU is
> +defined by the spec but its operation is OS dependent.  As of this writing 
> the
> +existing Linux kernel support `linux-v8`_, not yet merged, will configure 
> the IOMMU
> +to create IOMMU groups with any eligible cards available in the system,
> +regardless of factors such as the order in which the devices are added in the
> +command line.
> +
> +This means that these command lines are equivalent as far as the current
> +IOMMU kernel driver behaves:
> +
> +.. code-block:: bash
> +
> +  $ qemu-system-riscv64 \
> +        -M virt,aia=aplic-imsic,aia-guests=5 \
> +        -device riscv-iommu-pci,addr=1.0,vendor-id=0x1efd,device-id=0xedf1 \
> +        -device e1000e,netdev=net1 -netdev user,id=net1,net=192.168.0.0/24 \
> +        -device e1000e,netdev=net2 -netdev user,id=net2,net=192.168.200.0/24 
> \
> +        (...)
> +
> +  $ qemu-system-riscv64 \
> +        -M virt,aia=aplic-imsic,aia-guests=5 \
> +        -device e1000e,netdev=net1 -netdev user,id=net1,net=192.168.0.0/24 \
> +        -device e1000e,netdev=net2 -netdev user,id=net2,net=192.168.200.0/24 
> \
> +        -device riscv-iommu-pci,addr=1.0,vendor-id=0x1efd,device-id=0xedf1 \
> +        (...)
> +
> +Both will create iommu groups for the two e1000e cards.
> +
> +Another thing to notice on `linux-v8`_ is that the kernel driver considers 
> an IOMMU
> +identified as a Rivos device, i.e. it uses Rivos vendor ID.  To use the 
> riscv-iommu-pci
> +device with the existing kernel support we need to emulate a Rivos PCI IOMMU 
> by
> +setting 'vendor-id' and 'device-id':
> +
> +.. code-block:: bash
> +
> +  $ qemu-system-riscv64 -M virt        \
> +     -device riscv-iommu-pci,vendor-id=0x1efd,device-id=0xedf1 (...)
> +
> +Several options are available to control the capabilities of the device, 
> namely:
> +
> +- "bus": the bus that the IOMMU device uses
> +- "ioatc-limit": size of the Address Translation Cache (default to 2Mb)
> +- "intremap": enable/disable MSI support
> +- "ats": enable ATS support
> +- "off" (Out-of-reset translation mode: 'on' for DMA disabled, 'off' for 
> 'BARE' (passthrough))
> +- "s-stage": enable s-stage support
> +- "g-stage": enable g-stage support
> +
> +.. _iommu1.0: 
> https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf
> +
> +.. _linux-v8: 
> https://lore.kernel.org/linux-riscv/cover.1718388908.git.tjeznach@rivosinc.com/
> diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
> index c3f94f63da..dd884ea919 100644
> --- a/docs/system/riscv/virt.rst
> +++ b/docs/system/riscv/virt.rst
> @@ -84,6 +84,19 @@ none``, as in
>
>  Firmware images used for pflash must be exactly 32 MiB in size.
>
> +riscv-iommu support
> +-------------------
> +
> +The board has support for the riscv-iommu-pci device by using the following
> +command line:
> +
> +.. code-block:: bash
> +
> +  $ qemu-system-riscv64 -M virt -device riscv-iommu-pci (...)
> +
> +Refer to :ref:`riscv-iommu` for more information on how the RISC-V IOMMU 
> support
> +works.
> +
>  Machine-specific options
>  ------------------------
>
> --
> 2.45.2
>
>