Re: [Qemu-devel] [RFC v2 0/8] VIRTIO-IOMMU device

[Tian, Kevin]

> From: Jean-Philippe Brucker [mailto:address@hidden
> Sent: Friday, July 7, 2017 11:15 PM
> 
> On 07/07/17 07:21, Tian, Kevin wrote:
> > sorry I didn't quite get this part, and here is my understanding:
> >
> > Guest programs vIOMMU to map a gIOVA (used by MSI to a GPA
> > of doorbell register of virtual irqchip. vIOMMU then
> > triggers VFIO map/unmap to update physical IOMMU page
> > table for gIOVA -> HPA of real doorbell of physical irqchip
> 
> At the moment (non-SVM), physical and virtual MSI doorbell are completely
> dissociated. VFIO itself maps the doorbell GPA->HPA during container
> initialization. The GPA, chosen arbitrarily by the host, is then removed
> from the guest GPA space.

got you. I also got some basic understanding from below link. :-)

https://www.linaro.org/blog/core-dump/kvm-pciemsi-passthrough-armarm64/

> 
> When the guest programs the vIOMMU to map a gIOVA to the virtual irqchip
> doorbell, I suppose Qemu will notice that the GPA doesn't correspond to
> RAM and will withhold sending a VFIO_IOMMU_MAP_DMA request.
> 
> (For SVM I don't want to go into the details just now, but we will
> probably need a separate VFIO mechanism to update the physical MSI-X
> tables with whatever gIOVA the guest mapped in its private stage-1 page
> tables.)

I guess there may be either a terminology difference or a hardware
difference here, since I noted you mentioned IOVA with stage-1
multiple times.

For Intel VT-d:

- stage-1 is only for VA translation, tagged with PASID
- stage-2 can be used for IOVA translation on bare metal or GPA/gIOVA
translation in virtualization, w/o PASID tagged

Does ARM SMMU allow stage-1 used for both VA and IOVA? IIRC
you said PASID#0 reserved for traffic w/o PASID in some mail...

> 
> > (assume your irqchip will provide multiple doorbells so each
> > device can have its own channel).
> 
> In existing irqchips the doorbell is shared by endpoints, which are
> differentiated by their device ID (generally the BDF). I'm not sure why
> this matters here?

Not matter now with device ID

> 
> > then once this update is
> > done, later MSI interrupts from assigned device will go
> > through physical IOMMU (gIOVA->HPA) then reach irqchip
> > for irq remapping. vIOMMU is involved only in configuration
> > path instead of actual interrupt path.
> 
> Yes the vIOMMU is used to correlate the IOVA written by the guest in its
> virtual MSI-X table with the MAP request received by the vIOMMU. That is
> probably used to setup IRQFD routes with KVM. But the vIOMMU is not
> involved further than that in MSIs.
> 
> > If my understanding is correct, above will be the natural flow then
> > why is additional virtio-iommu change required? :-)
> 
> The change is not *required* for ARM systems, I only proposed removing the
> doorbell address translation stage to make host implementation simpler
> (and since virtio-iommu on x86 won't translate the doorbell anyway, we
> have to add support for this to virtio-iommu). But for Qemu, since vSMMU
> needs to implement the natural flow anyway, it might not be a lot of
> effort to also do it for virtio-iommu. Other implementations (e.g.
> kvmtool) might piggy-back on the x86 way and declare the irqchip doorbell
> as untranslated.
> 
> My proposal also breaks when confronted to virtual SVM in a physical ARM
> system, where the guest owns stage-1 page tables and *has* to map the
> doorbell if it wants MSIs to work, so you can disregard it :)
> 

It is a good learning. thanks.