Re: [Qemu-devel] [SeaBIOS] [RFC v2 0/3] Support multiple pci domains in pci_device

[Gerd Hoffmann]

On Tue, Aug 28, 2018 at 04:12:17AM +0000, Zihan Yang wrote:
> Gerd Hoffmann <address@hidden> 于2018年8月27日周一 上午7:04写道：
> >
> >   Hi,
> >
> > > >   However, QEMU only binds port 0xcf8 and 0xcfc to
> > > > bus pcie.0. To avoid bus confliction, we should use other port pairs for
> > > > busses under new domains.
> > >
> > > I would skip support for IO based configuration and use only MMCONFIG
> > > for extra root buses.
> > >
> > > The question remains: how do we assign MMCONFIG space for
> > > each PCI domain.
> >
> > Allocation-wise it would be easiest to place them above 4G.  Right after
> > memory, or after etc/reserved-memory-end (if that fw_cfg file is
> > present), where the 64bit pci bars would have been placed.  Move the pci
> > bars up in address space to make room.
> >
> > Only problem is that seabios wouldn't be able to access mmconfig then.
> >
> > Placing them below 4G would work at least for a few pci domains.  q35
> > mmconfig bar is placed at 0xb0000000 -> 0xbfffffff, basically for
> > historical reasons.  Old qemu versions had 2.75G low memory on q35 (up
> > to 0xafffffff), and I think old machine types still have that for live
> > migration compatibility reasons.  Modern qemu uses 2G only, to make
> > gigabyte alignment work.
> >
> > 32bit pci bars are placed above 0xc0000000.  The address space from 2G
> > to 2.75G (0x8000000 -> 0xafffffff) is unused on new machine types.
> > Enough room for three additional mmconfig bars (full size), so four
> > pci domains total if you add the q35 one.
> 
> Maybe we can support 4 domains first before we come up
> with a better solution. But I'm not sure if four domains are
> enough for those who want too many devices?

There is the option to use ovmf instead.  Runs in long mode, so mapping
and using mmconfig above 4G isn't a problem for ovmf.  Given that
alternative exists I think it is reasonable to declare only 4 pci
domains being fully supported with seabios.

We can even configure mmconfig (above 4G) for the 5th and above, seabios
just would not enumerate the devices and map the pci bars for these
domains and leave it to the guest os instead.

cheers,
  Gerd