Re: [PATCH v6 12/19] vfio-user: IOMMU support for remote device

[Stefan Hajnoczi]

On Mon, Mar 07, 2022 at 02:42:49PM +0000, Jag Raman wrote:
> 
> 
> > On Mar 7, 2022, at 4:45 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > 
> > On Thu, Mar 03, 2022 at 02:49:53PM +0000, Jag Raman wrote:
> >> 
> >> 
> >>> On Mar 2, 2022, at 11:49 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >>> 
> >>> On Mon, Feb 28, 2022 at 07:54:38PM +0000, Jag Raman wrote:
> >>>> 
> >>>> 
> >>>>> On Feb 22, 2022, at 5:40 AM, Stefan Hajnoczi <stefanha@redhat.com> 
> >>>>> wrote:
> >>>>> 
> >>>>> On Thu, Feb 17, 2022 at 02:48:59AM -0500, Jagannathan Raman wrote:
> >>>>>> +struct RemoteIommuElem {
> >>>>>> +    AddressSpace  as;
> >>>>>> +    MemoryRegion  mr;
> >>>>>> +};
> >>>>>> +
> >>>>>> +GHashTable *remote_iommu_elem_by_bdf;
> >>>>> 
> >>>>> A mutable global hash table requires synchronization when device
> >>>>> emulation runs in multiple threads.
> >>>>> 
> >>>>> I suggest using pci_setup_iommu()'s iommu_opaque argument to avoid the
> >>>>> global. If there is only 1 device per remote PCI bus, then there are no
> >>>>> further synchronization concerns.
> >>>> 
> >>>> OK, will avoid the global. We would need to access the hash table
> >>>> concurrently since there could be more than one device in the
> >>>> same bus - so a mutex would be needed here.
> >>> 
> >>> I thought the PCIe topology can be set up with a separate buf for each
> >>> x-vfio-user-server? I remember something like that in the previous
> >>> revision where a root port was instantiated for each x-vfio-user-server.
> >> 
> >> Yes, we could setup the PCIe topology to be that way. But the user could
> >> add more than one device to the same bus, unless the bus type explicitly
> >> limits the number of devices to one (BusClass->max_dev).
> > 
> > Due to how the IOMMU is used to restrict the bus to the vfio-user
> > client's DMA mappings, it seems like it's necesssary to limit the number
> > of devices to 1 per bus anyway?
> 
> Hi Stefan,
> 
> “remote_iommu_elem_by_bdf” has a separate entry for each of the BDF
> combinations - it provides a separate DMA address space per device. As
> such, we don’t have to limit the number of devices to 1 per bus.

I see, thanks!

Stefan

signature.asc
Description: PGP signature