Re: [Qemu-devel] [PATCH] intel_iommu: make sure its init before PCI dev

[Alex Williamson]

On Thu, 23 Feb 2017 14:02:07 +0200
Marcel Apfelbaum <address@hidden> wrote:

> On 02/23/2017 10:16 AM, Peter Xu wrote:
> > On Thu, Feb 23, 2017 at 10:10:57AM +0200, Marcel Apfelbaum wrote:  
> >> On 02/23/2017 05:06 AM, Peter Xu wrote:  
> >>> On Wed, Feb 22, 2017 at 10:30:47AM -0700, Alex Williamson wrote:  
> >>>> On Wed, 22 Feb 2017 13:49:25 +0800
> >>>> Peter Xu <address@hidden> wrote:
> >>>>  
> >>>>> Intel vIOMMU devices are created with "-device" parameter, while here
> >>>>> actually we need to make sure this device will be created before some
> >>>>> other PCI devices (like vfio-pci devices) so that we know iommu_fn will
> >>>>> be setup correctly before realizations of those PCI devices.
> >>>>>
> >>>>> Here we do explicit check to make sure intel-iommu device will be inited
> >>>>> before all the rest of the PCI devices. This is done by checking against
> >>>>> the devices dangled under current root PCIe bus and we should see
> >>>>> nothing there besides integrated ICH9 ones.  
> >>
> >> Hi,
> >>
> >> Commit b86eacb8 (hw/pci: delay bus_master_enable_region initialization)
> >> creates the IOMMU memory region at machine_done time so the
> >> devices creation order wouldn't matter.
> >>
> >> I don't think we use the iommu_fn before machine_done.
> >> What have I missed?  
> >
> > Hi, Marcel,
> >
> > The problem is that vfio-pci will need to fetch the pci address space
> > during realization (pci_device_iommu_address_space() is called in
> > vfio_realize()).
> >
> > Any thoughts? Thanks,  
> 
> Sure, I'll try to find a solution, but first I need to
> understand why vfio-pci need to access the pci_device_iommu_address_space
> so early. Maybe we can delay this also to machine_done stage.

It's the architecture of vfio, the user only gets access to the device
when the container has iommu protection, therefore vfio needs to look
at the device address space to determine if it can share a container
with other devices.  Without an iommu all devices share the system
address space and use the same container.  With an iommu, each device
is in a separate address space and each gets its own container.
Without a container, the user doesn't get access to the device.
Deferring the address space to machine done would essentially defer the
entire vfio device initialization or else we'd need to close the
device and re-open and initialize it through a new container at that
time.  Thanks,

Alex