Re: [Qemu-devel] [PATCH] ppc/pnv: Add model for Power8 PHB3 PCIe Host bridge

[David Gibson]

On Thu, Jun 28, 2018 at 10:14:53PM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2018-06-28 at 10:00 +0200, Andrea Bolognani wrote:
> > On Thu, 2018-06-28 at 13:59 +1000, David Gibson wrote:
> > > On Wed, Jun 27, 2018 at 12:22:31PM +0200, Andrea Bolognani wrote:
> > > > On Tue, 2018-06-26 at 19:02 +0200, Cédric Le Goater wrote:
> > > > > I didn't follow that discussion but this is "another" kind of PHB.
> > > > > This one models the baremetal controller as found on OpenPOWER and
> > > > > IBM Power machines. pSeries has a virtual PHB.
> > > > 
> > > > I understand that, and of course libvirt will need to learn about
> > > > this new type of PHB and make sure both pSeries and PowerNV guests
> > > > get the correct one assigned to them.
> > > 
> > > Hmm.. does it?  I would have thought pnv could act more like x86, in
> > > that libvirt doesn't attempt to create PHBs at all and just use the
> > > ones that are built in.
> > 
> > AFAIK x86 guests have a single PHB and additional ones cannot be
> > created in any way, which means we don't have to do any additional
> > second-guessing when assigning IDs to additional PCI controllers.
> 
> That's a surprising limitation. A single PHB only supports a limited
> number of MSIs no ? And only 256 bus numbers...

I think it depends exactly what you call a "PHB".  AIUI, on modern x86
systems, multiple PCI domains are supported, but you access them all
through the same IO ports, using a 'domain' field in some register to
distinguish which you're operating on

Wheter you want to call that multiple PHBs with a register multiplexer
in front of them, or a single PHB off which hang multiple domains is
kind of arbitrary (at least from the guest PoV).

> > > Though, come to that, I wouldn't think pnv support for libvirt would
> > > be much of a priority anyway.  The machine type is still very much in
> > > flux, and it's designed primarily for testing and development, not
> > > "real world" usage.
> > 
> > Can you *guarantee* that someone won't ask for PowerNV support in
> > libvirt at some point? Because if you can't (and I don't think you
> > can ;) then this is still a valuable conversation to have.
> 
> It's rather unlikely for now as there is no KVM suport for it (it's
> tricky, our chips aren't designed for full virtualization). That might
> change in the future but not soon.

KVM support isn't really a prerequisite for libvirt support.  More
relevant is that the qemu level machine is still changing a lot.  I
don't believe we're really maintaining version to version option
compatibility at this point, we're certainly not attempting to support
cross version migration for it.

> > > > What I meant is that pSeries guests get a single PHB by default,
> > > > with additional ones being instantiable through -device; this is
> > > > also consistent with how PCI controllers are added to other guest
> > > > types including pc, q35 and aarch64/virt, so it would be really
> > > > nice if PowerNV behaved the same way.
> > > 
> > > Well.. sure.. but it doesn't.  pSeries is a virtual platform, so we
> > > have a reasonable amount of flexibility to define it as we want.
> > > PowerNV is an emulation of existing hardware which has a specific
> > > behaviour which we need to match.
> > 
> > Sure, that's something to keep in mind.
> > 
> > But the thing is, you still need to have *some* flexibility in
> > the number of PHBs, since there is variation among real Power8
> > and Power9 chips; in the current incarnation, that flexibility
> > is provided by the num_phbs parameter, which is an entirely new
> > interface that's exclusive to PowerNV.
> > 
> > What I'm suggesting is that the same amount of flexibility is
> > offered through a standard interface, namely -device, instead.
> 
> But that's harder internally to qemu to properly "bind" to the chip
> where the PHB resides etc... 
> 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature