Re: [Qemu-devel] [PATCH RFC] docs: add PCIe devices placement guidelines

[Laine Stump]

On 09/07/2016 04:06 AM, Marcel Apfelbaum wrote:
> On 09/07/2016 09:21 AM, Gerd Hoffmann wrote:
> >   Hi,
> >
> > > > > ports, if that's allowed). For example:
> > > > >
> > > > > -  1-32 ports needed: use root ports only
> > > > >
> > > > > - 33-64 ports needed: use 31 root ports, and one switch with 2-32
> > > > > downstream ports
> >
> > I expect you rarely need any switches.  You can go multifunction with
> > the pcie root ports.  Which is how physical q35 works too btw, typically
> > the root ports are on slot 1c for intel chipsets:
> >
> > nilsson root ~# lspci -s1c
> > 00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset
> > Family PCI Express Root Port 1 (rev c4)
> > 00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset
> > Family PCI Express Root Port 2 (rev c4)
> > 00:1c.2 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset
> > Family PCI Express Root Port 3 (rev c4)
> >
> > Root bus has 32 slots, a few are taken (host bridge @ 00.0, lpc+sata @
> > 1f.*, pci bridge @ 1e.0, maybe vga @ 01.0), leaving 28 free slots.  With
> > 8 functions each you can have up to 224 root ports without any switches,
> > and you have not many pci bus numbers left until you hit the 256 busses
> > limit ...
>
> Good point, maybe libvirt can avoid adding switches unless the user
> explicitly
> asked for them. I checked and it a actually works fine in QEMU.

I'm just now writing the code that auto-adds *-ports as they are needed, 
and doing it this way simplifies it *immensely*.

When I had to think about the possibility of needing upstream/downstream 
switches, as an endpoint device was added, I would need to check if a 
(root|downstream)-port was available and if not I might be able to just 
add a root-port, or I might have to add a downstream-port; if the only 
option was a downstream port, then *that* might require adding a new 
*upstream* port.

If I can limit libvirt to only auto-adding root-ports (and if there is 
no downside to putting multiple root ports on a single root bus port), 
then I just need to find an empty function of an empty slot on the root 
bus, add a root-port, and I'm done (and since 224 is *a lot*, I think at 
least for now it's okay to punt once they get past that point).

So, *is* there any downside to doing this?