Re: [Qemu-devel] [RFC] virtio-pci: Allow PCIe virtio devices on root bus

[Marcel Apfelbaum]

On 02/16/2017 04:48 AM, David Gibson wrote:
> On Wed, Feb 15, 2017 at 04:59:33PM +0200, Marcel Apfelbaum wrote:

[...]

> > > > > I did float the idea of having the pseries PCI bus remain plain PCI
> > > > > but with a special flag to allow PCIe devices to be attached to it
> > > > > anyway.  It wasn't greeted with much enthusiasm..
> > > >
> > > > Can you point me to the discussion please? It seems similar to what I proposed 
> > > > above.
> > >
> > > Sorry, I was misleading.  I think I just raised that idea with Andrea
> > > and a few other people internally, not on one of the lists at large.
> > >
> > > > As you properly described it, is much closer to PCI then PCIe, even the only 
> > > > characteristic
> > > > that makes it "a little" PCIe, the Extended Configuration Space support,
> > > > is done with an alternative interface.
> > > >
> > > > I agree the PAPR bus is not PCIe.
> > >
> > > Ok, so if we take that direction, the question becomes how do we let
> > > PCIe devices plug into this mostly-not-PCIe bus.  Maybe introduce a
> > > "pci_bus_accepts_express()" function that will replace many, but not
> > > all current uses of "pci_bus_is_express()"?
> >
> > Sounds good and I think Eduardo is already working on exactly this
> > idea, however he is on PTO now. It is better to synchronize with him.
>
> Ah, right.  Do you know when he'll be back?  This is semi-urgent for
> Power.

In a week or so.

>

[...]

> > > Ok.. I'm still missing something.  Are you saying that instead of the
> > > legacy/modern status determining PCIe support, that PCIe status is
> > > determining legacy/modern support by default?
> >
> > Is more a guideline. We want QEMU to behave like this *by default*.
> > The modern spec does not require virtio 1.0 devices to be PCIe.
>
> Ok.  But I'm trying to get a handle on how - even by default - the
> linkage between PCI/PCIe and modern/legacy is created:  it's not
> obvious to me.

There is no direct connection between modern/legacy <-> PCI/PCIe.
However the defaults work like that:
I440FX - (pc): devices are PCI, transitional
   created by: disable-legacy=auto,disable-modern=off
   - in virtio_pci_realize disable-legacy: auto => off
Q35 - on root bus - devices are PCI, transitional
    - other buses - devices are PCIe, modern
   created by: disable-legacy=auto,disable-modern=off
   - in virtio_pci_dc_realize:
        if disable-modern=off -> pcie
   - in virtio_pci_realize:
    if on pcie root port -> disable-legacy: auto => on
    otherwise ->    disable-legacy: auto => off


> >   That would actually
> > > seem to simplify things: whatever method we end up allowing PCIe
> > > virtio on PAPR, that should automatically enable modern mode, which is
> > > fine.
> >
> > Agreed.
> >
> > > Although.. I do wonder about legacy/modern for PAPR in general.
> > > Current kernels will support virtio 1.0 fine, but we have older
> > > released versions which only support legacy.
> >
> > So, do you want legacy virtio devices to be PCIe ?
>
> TBH, we probably don't care one way or another.
>
> I'm not clear here if you're making a distinction between legacy-only
> and legacy+modern virtio instances.

Legacy devices and the modern ones are different devices (product ID).
I think the 'transitional' ones have the ID of the modern virtio.

> I think we need to keep legacy support on for Power machines, because
> we don't have the machine type switchover to let us handle it.  But we
> certainly would like to enable modern support so that newer guests can
> exploit it.

"transitional" is the right way to go.

Thanks,
Marcel

> >   Unlike PC we won't (I
> > > hope) have a whole machine type switch to handle this.
> >
> > Good for you!
> >
> >   At this stage
> > > I think we want virtio devices (whatever their bus type) on PAPR to
> > > default to allowing both legacy and modern for the forseeable future.
> > > How does that affect the matrix?
> >
> > It adds a legacy virtio PCIe device to the matrix. For X86 we don't need
> > this combination because PC machines are PCI and Q35 machines are to be used
> > only on modern setups.
> >
> > Since PARP does not have separate machines for PCi and PCIe, we need to live 
> > with it.
> >
> >
> > > > XHCI being PCIe on Root Complex is an unintended exception, but we want it 
> > > > connected to a
> > > > Root Port anyway, we don't have anything to gain from having it as Integrated 
> > > > Endpoint.
> > > > We only loose a slot that can be used by 8 Root Ports assembled into one 
> > > > multi-function device.
> > > >
> > > > PAPR bus should not be considered PCIe and should have a different set of rules 
> > > > allowing PCIe
> > > > devices to be plugged into Root Complex.
> > >
> > > Alright, I can work with that.  Michael, Andrea, how does this idea
> > > seem to you?
> >
> > We should definitely get more opinions.
> > Thanks,
> > Marcel