Re: [Qemu-devel] vfio for platform devices - 9/5/2012 - minutes

[Alexander Graf]

On 12.09.2013, at 13:10, Scott Wood wrote:

> On Thu, 2013-09-12 at 04:18 -0500, Bhushan Bharat-R65777 wrote:
>> 
>>> -----Original Message-----
>>> From: Alex Williamson [mailto:address@hidden
>>> Sent: Wednesday, September 11, 2013 10:45 PM
>>> To: Yoder Stuart-B08248
>>> Cc: Wood Scott-B07421; Sethi Varun-B16395; Bhushan Bharat-R65777; 'Peter
>>> Maydell'; 'Santosh Shukla'; 'Alexander Graf'; 'Antonios Motakis'; 
>>> 'Christoffer
>>> Dall'; 'address@hidden'; address@hidden; kvm-
>>> address@hidden; address@hidden
>>> Subject: Re: vfio for platform devices - 9/5/2012 - minutes
>>> 
>>> On Wed, 2013-09-11 at 16:42 +0000, Yoder Stuart-B08248 wrote:
>>>> 
>>>>> -----Original Message-----
>>>>> From: Yoder Stuart-B08248
>>>>> Sent: Thursday, September 05, 2013 12:51 PM
>>>>> To: Wood Scott-B07421; Sethi Varun-B16395; Bhushan Bharat-R65777;
>>>>> 'Peter Maydell'; 'Santosh Shukla'; 'Alex Williamson'; 'Alexander
>>>>> Graf'; 'Antonios Motakis'; 'Christoffer Dall'; 'address@hidden'
>>>>> Cc: address@hidden; 'address@hidden'; 'qemu-
>>>>> address@hidden'
>>>>> Subject: vfio for platform devices - 9/5/2012 - minutes
>>>>> 
>>>>> We had a call with those interested and/or working on vfio for
>>>>> platform devices.
>>>>> 
>>>>> Participants: Scott Wood, Varun Sethi, Bharat Bhushan, Peter Maydell,
>>>>>              Santosh Shukla, Alex Williamson, Alexander Graf,
>>>>>              Antonios Motakis, Christoffer Dall, Kim Phillips,
>>>>>              Stuart Yoder
>>>>> 
>>>>> Several aspects to vfio for platform devices:
>>>>> 
>>>>> 1. IOMMU groups
>>>>> 
>>>>> -iommu driver needs to register a bus notifier for the platform bus
>>>>>  and create groups for relevant platform devices  -Antonios is
>>>>> looking at this for several ARM IOMMUs  -PAMU (Freescale) driver
>>>>> already does this
>>>>> 
>>>>> 2. unbinding device from host
>>>>> 
>>>>> PCI:
>>>>>   echo 0000:06:0d.0 >
>>>>> /sys/bus/pci/devices/0000:06:0d.0/driver/unbind
>>>>> Platform:
>>>>>   echo ffe101300.dma >
>>>>> /sys/bus/platform/devices/ffe101300.dma/driver/unbind
>>>>> 
>>>>> -don't believe there are issues or work to do here
>>>>> 
>>>>> 3. binding device to vfio-platform driver
>>>>> 
>>>>> PCI:
>>>>>   echo 1102 0002 > /sys/bus/pci/drivers/vfio-pci/new_id
>>>>> 
>>>>> -this is probably the least understood issue-- platform drivers
>>>>>  register themselves with the bus for a specific "name"
>>>>>  string.  That is matched with device tree "compatible" strings
>>>>>  later to bind a device to a driver  -we want is to have the
>>>>> vfio-platform driver to dynamically bind
>>>>>  to a variety of platform devices previously unknown to
>>>>>  vfio-platform
>>>>> -ideally unbinding and binding could be an atomic operation  -Alex
>>>>> W pointed out that x86 could leverage this work so
>>>>>  keep that in mind in what we design  -Kim Phillips (Linaro) will
>>>>> start working on this
>>>> 
>>>> One thing we didn't discuss needs to be considered (probably by Kim
>>>> who is looking at the 'binding device' issue) is around returning a
>>>> passthru device back to the host.
>>>> 
>>>> After a platform device has been bound to vfio and is in use by user
>>>> space or a virtual machine, we also need to be able to unwind all that
>>>> and return the device back to the host in a sane state.
>>>> 
>>>> What happens when user space exits and the vfio file descriptors are
>>>> closed?
>>> 
>>> For reference, expectations of how vfio-pci handles these situations:
>>> 
>>> For vfio-pci, when the reference count on the device fd drops to zero we 
>>> call a
>>> device disable function that includes disabling the bus master bit in config
>>> space stop ongoing DMA.
>> 
>> There is no bus mastering for platform devices
> 
> Sure there is -- that's just PCI's name for DMA.
> 
>> and device disabling is not a standard like PCI. Do you think that we might 
>> need to do some
>> device specific handling.
> 
> It would be better if we could avoid the need for device specific
> handling.  DMA can be disabled via the IOMMU.
> 
> The tricky part is how to enable the device in the next user; as I wrote
> elsewhere in the thread I think this needs to be something the user is
> aware of, so that the user quiesces the device prior to enabling IOMMU
> mappings.

Or resets it. I think we can make it mandatory to VFIO users (and Linux 
drivers) to reset the device prior to mapping it. QEMU needs to have device 
specific code to assemble its device tree fragment anyways, so it can just as 
well reset the device before it enables it to DMA.

Alex