Re: [Qemu-devel] [PATCH v9 05/11] vfio: add check host bus reset is support or not

[Dou Liyang]

Hi Alex,

At 09/22/2016 10:03 PM, Alex Williamson wrote:
> On Thu, 22 Sep 2016 16:34:32 +0800
> Dou Liyang <address@hidden> wrote:
>
> > Hi Alex,
> >
> > At 09/01/2016 03:56 AM, Alex Williamson wrote:
> > > On Tue, 19 Jul 2016 15:38:23 +0800
> > > Zhou Jie <address@hidden> wrote:
> > >
> > > > From: Chen Fan <address@hidden>
> > > >
> > > > When assigning a vfio device with AER enabled, we must check whether
> > > > the device supports a host bus reset (ie. hot reset) as this may be
> > > > used by the guest OS in order to recover the device from an AER
> > > > error.  QEMU must therefore have the ability to perform a physical
> > > > host bus reset using the existing vfio APIs in response to a virtual
> > > > bus reset in the VM.  A physical bus reset affects all of the devices
> > > > on the host bus, therefore we place a few simplifying configuration
> > > > restriction on the VM:
> > > >
> > > >  - All physical devices affected by a bus reset must be assigned to
> > > >    the VM with AER enabled on each and be configured on the same
> > > >    virtual bus in the VM.
> > > >
> > > >  - No devices unaffected by the bus reset, be they physical, emulated,
> > > >    or paravirtual may be configured on the same virtual bus as a
> > > >    device supporting AER signaling through vfio.
> > > >
> > > > In other words users wishing to enable AER on a multifunction device
> > > > need to assign all functions of the device to the same virtual bus
> > > > and enable AER support for each device.  The easiest way to
> > > > accomplish this is to identity map the physical functions to virtual
> > > > functions with multifunction enabled on the virtual device.
> > >
> > > Why am I able to start the following VM with aer=on for the vfio-pci
> > > devices?
> >
> > In my opinion, because, when we check out this situation in vfio_init,
> > our code just do nothing except going to out_teardown. so the devices
> > which are behind a PCIe-to-PCI bridge will be regarded as the PCI devices.
> >
> > > # lspci -tv
> > > -[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
> > >            +-01.0  Device 1234:1111
> > >            +-1c.0-[01]--
> > >            +-1d.0-[02]--+-01.0  Intel Corporation 82576 Gigabit Network 
> > > Connection
> > >            |            \-01.1  Intel Corporation 82576 Gigabit Network 
> > > Connection
> > >            ...
> > >
> > > # lspci -vvv -s 1d.0
> > > 00:1d.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge (prog-if 00 [Normal 
> > > decode])
> > >
> > > The devices are behind a PCIe-to-PCI bridge, so shouldn't specifying
> > > aer=on for the vfio-pci devices cause a configuration error?
> >
> > Yes, I think it may makes users confuse.
> >
> > I added a error_report for it.
>
> I think we need more than an error_report, our strategy to date has
> been that if the user specifies aer=on and we cannot support AER on the
> device then we fail the initialization of the device.

Oh, I am sorry, I did not say clearly. I also made the init failure by
"return -EINVAL".

 Eric's patches
> to convert vfio to realize and overhaul the error handling might make
> this easier to accomplish.  Thanks,

Yes, I saw  Eric's patches too.

I will follow Eric's way.

Thanks very much.
Dou


> Alex
>
> > Then, I test many passible combinations, such as:
> >
> > pci-bridge
> >      |- vfio-pci
> >
> > -device pci-bridge,chassis_nr=3,id=pci.3,bus=pcie.0,addr=0x1d \
> > -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.3,addr=1 \
> >
> > error:
> > vfio: Unable to enable AER for device 0000:06:00.0, device does not
> > support AER signaling
> >
> > --------------------------------------------------------------------
> >
> > pci-bridge
> >      |- vfio-pci
> >      |- vfio-pci
> >
> > -device pci-bridge,chassis_nr=3,id=pci.3,bus=pcie.0,addr=0x1d \
> > -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.3,addr=1 \
> > -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=pci.3,addr=1.1 \
> >
> > error:
> > vfio: Unable to enable AER for device 0000:06:00.0, device does not
> > support AER signaling
> >
> > --------------------------------------------------------------------
> > ioh3420
> >    |- pci-bridge
> >        |- vfio-pci
> >        |- vfio-pci
> >
> > -device ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1,chassis=1 \
> > -device pci-bridge,chassis_nr=3,id=pci.1,bus=bridge1,addr=0x1d \
> > -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.1,addr=1 \
> > -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=pci.1,addr=1.1 \
> >
> > error:
> > vfio: Unable to enable AER for device 0000:06:00.0, device does not
> > support AER signaling
> >
> > --------------------------------------------------------------------
> > just for test:
> > pci-bridge
> >    |- ioh3420
> >        |- vfio-pci
> >        |- vfio-pci
> > -device pci-bridge,chassis_nr=3,id=pci.1,bus=pcie.0,addr=0x1d \
> > -device ioh3420,bus=pci.1,addr=1c.0,port=1,id=bridge1,chassis=1 \
> > -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=bridge1,addr=1 \
> > -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=bridge1,addr=1.1 \
> >
> > error:
> > vfio: Unable to enable AER for device 0000:06:00.0, parent bus does not
> > support AER signaling
> >
> > --------------------------------------------------------------------
> > ioh3420
> >     |- vfio-pci
> >     |- vfio-pci
> >
> > -device ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1,chassis=1 \
> > -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=bridge1,addr=1 \
> > -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=bridge1,addr=1.1 \
> >
> > OK
> > --------------------------------------------------------------------
> > ...
> >
> > Thanks,
> > Dou
> >
> > >
> >
> > [...]
> >
> >
> > >