Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'

[Markus Armbruster]

Eduardo Habkost <address@hidden> writes:

> On Thu, Nov 24, 2016 at 02:34:05PM +0100, Markus Armbruster wrote:
>> Eduardo Habkost <address@hidden> writes:
>> 
>> > On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
>> >> On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
>> >> > The Problem
>> >> > ===========
>> >> > 
>> >> > Currently management software has no way to find out which device
>> >> > types can be plugged in a machine, unless the machine is already
>> >> > initialized.
>> 
>> I doubt it can find the precise list of device types even for an
>> initialized machine.
>
> My plan is to make it possible. See:
>   [RFC 06/15] qdev: Add device_type field to BusClass
>
> I will change how it looks like on v2 (mostly due to the PCI/PCIe
> issues), but basically it makes all bus objects able to tell
> which device types they accept.

Sounds like a step forward.

>> >> Hi Eduardo,
>> >> Thank you for this interesting series. I think this is a problem
>> >> worth addressing.
>> >> 
>> >> > Even after the machine is initialized, there's no way to map
>> >> > existing bus types to supported device types unless management
>> >> > software hardcodes the mapping between bus types and device
>> >> > types.
>> >> > 
>> >> 
>> >> Here I am a little lost.
>> >> 
>> >> We are going for machine => supported devices or
>> >> bus-type => supported devices?
>> >
>> > On this series, we go for machine-type => supported-devices.
>> >
>> > A bus-type => supported-devices map wouldn't work because
>> > different PCIe bus instances might accept different types of
>> > devices (so supported-devices depend on the specific bus
>> > instance, not only on the bus-type).
>> >
>> > v2 will probably be more detailed. I plan to change it to:
>> >
>> > query-machine(machine-type) => list of BusInfo
>> >
>> > BusInfo would contain:
>> >  * bus-type
>> >  * bus-path
>> >  * accepted-device-types (list of type/interface names)
>> 
>> Let me take a step back and explore what questions a management
>> application may want to ask.  I think the basic questions are:
>> 
>> * Which devices models could be plugged now?
>> 
>>   "Now" means in the current state of the machine.  Depending on "now",
>>   this could be hot- or cold-plug.
>
> Yes.
>
>> 
>> * Into which sockets could device model T be plugged now?
>> 
>>   Mind, I wrote "socket", not bus.  Buses provide sockets, but they are
>>   not the only provider of sockets.
>> 
>>   We need a way to name sockets.  (QOM path to device or bus, socket
>>   address on device or bus) could work.
>
> Naming "sockets" was not on my model, but I think we can do that.
>
> I will probably send RFC v2 without any socket abstraction, so
> other conceptual changes I am introducing can be reviewed
> (especially the PCI/PCIe mess I am diving into, right now). But I
> will try to document how the design can evolve to handle sockets.

Makes sense.

>> Are we on the same page so far?
>
> Yes.
>
>> 
>> Related: same questions with "newly created machine of machine type M"
>> instead of "now".  This isn't basic, because you could always create the
>> machine, then ask the basic question.  But not having to create the
>> machine is convenient, in particular as long as a single QEMU process
>> can only ever create one machine.
>> 
>> Related: same, with creation options.
>> 
>> Your series provides a solution for a special non-basic case: newly
>> created machine of machine type M with no optional parts, i.e. with
>> -nodefaults and nothing else.  Or maybe with the default set of optional
>> parts, i.e. without -nodefaults, not sure.
>> 
>> I'm not demanding you provide a solution for the general case first.  It
>> would be nice, but it could be too complex for a first step.  I do
>> recommend you design your solution for the special case with the general
>> case in mind.  Say, by trying to ensure the special case query returns
>> something that will do or can be evolved to do for the general case,
>> too.
>
> Yep. I will keep that in mind.
>
>> 
>> >> > Example: floppy support on q35 vs i440fx
>> >> > ----------------------------------------
>> >> > 
>> >> > There's no way for libvirt to find out that there's no floppy
>> >> > controller on pc-q35-* machine-types by default.
>> >> > 
>> >> 
>> >> Again "by default". So do we want to query the init state of a machine?
>> >> What devices are there? Or what devices *can be* there?
>> >
>> > "by default" means what's present when using "-machine <machine>"
>> > with no extra -device arguments.
>> 
>> Not just -device.  Some legacy options can also create buses.  For
>> instance, -device if=scsi,... creates a SCSI bus.
>
> Right. I found that out when debugging the -nodefaults issues.
>
>> 
>> > We want to know what _buses_ are always there. Which in turn lets
>> > management know which _device_ types _can_ be plugged.
>> >
>> >> 
>> >> > With this series, pc-i440fx-* will report "floppy" as a supported
>> >> > device type, but pc-q35-* will not.
>> >> > 
>> >> > Example: Legacy PCI vs vs PCIe devices
>> >> > --------------------------------------
>> >> > 
>> >> > Some devices require a PCIe bus to be available, others work on
>> >> > both legacy PCI and PCIe, while others work only on a legacy PCI
>> >> > bus.
>> >> > 
>> >> > Currently management software has no way to know which devices
>> >> > can be added to a given machine, unless it hardcodes machine-type
>> >> > names and device-types names.
>> >> > 
>> >> 
>> >> Again it seems a double problem, machine => devices vs pci/pcie bus => 
>> >> devices.
>> >> The bus => devices match is not related to a machine type.
>> >
>> > A bus-type => device-type match would not depend on the
>> > machine-type, but it would not be useful: different bus instances
>> > can accept different device-types (and the way the bus topology
>> > is configured depend on the machine-type).
>> >
>> >
>> >> 
>> >> > The Proposed Interface
>> >> > ======================
>> >> > 
>> >> > This series adds a new field to the output of 'query-machines':
>> >> > 'supported-device-types'. It will contain a list of QOM type
>> >> > names, that can be used to find the list of device types that can
>> >> > be plugged in the machine by default.
>> >> 
>> >> What do you mean "by default"? Without bridges or part of the machine 
>> >> itself?
>> >
>> > I mean "when you just run -machine with no extra -device
>> > arguments".
>> >
>> >> 
>> >>  The type names reported on
>> >> > the new field can then be used as the 'implements' argument on
>> >> > the 'qom-list-types' command, to find out which device types can
>> >> > be plugged on the machine.
>> >> > 
>> >> > Example output
>> >> > --------------
>> >> > 
>> >> >   (QEMU) query-machines
>> >> >   {
>> >> >     "return": [
>> >> >         [...]
>> >> >         {
>> >> >             "supported-device-types": [
>> >> >                 "sys-bus-device"
>> >> 
>> >> 
>> >> I don't know how "sys-bus-device" can help us... :)
>> >
>> > Yes, I added comments about it below. :)
>> 
>> The sysbus has always been a cop-out.  It's a catch-all pseudo-bus.  The
>> only thing sysbus devices really have in common is that they can't plug
>> into any real bus.
>> 
>> This used to be an academic issue until Alex made some sysbus devices
>> pluggable.  Before, "sysbus device" told you everything about
>> pluggability you needed to know: not pluggable.  Since then, it tells
>> you nothing.  We'll have to address this regression eventually.  See
>> "TYPE_SYS_BUS_DEVICE is too generic" below.
>> 
>> >> >             ],
>> >> >             "cpu-max": 1,
>> >> >             "hotpluggable-cpus": false,
>> >> >             "name": "none"
>> >> >         },
>> >> >         [...]
>> >> >         {
>> >> >             "supported-device-types": [
>> >> >                 "sys-bus-device"
>> >> >             ],
>> >> >             "cpu-max": 1,
>> >> >             "hotpluggable-cpus": false,
>> >> >             "name": "xenpv"
>> >> >         },
>> >> >         [...]
>> >> >         {
>> >> >             "supported-device-types": [
>> >> >                 "sys-bus-device",
>> >> >                 "floppy",
>> >> >                 "i2c-slave",
>> >> >                 "pci-device",
>> >> >                 "isa-device",
>> >> >                 "ide-device"
>> >> 
>> >> Is don't know is this high level classification is useful,
>> >> here is an example:
>> >> 
>> >>    pvi-device is supported => then we look for all pci devices?
>> >> But what if some pci devices make sense on a machine type,
>> >> but not on another?
>> >
>> > If not all pci devices are supported, then the machine must not
>> > return "pci-device" as supported. We need to define a new
>> > type/interface name that would be implemented only by the
>> > supported devices. e.g. "legacy-pci-device".
>> 
>> Before PCIe, "PCI device" told you everything about pluggability you
>> needed to know: can be plugged into a PCI bus.  Since PCIe, it doesn't.
>> We'll have to address this regression, too.  See "PCI vs PCIe" below.
>
> I am trying to clean up the PCI/PCIe mess right now, for v2.
>
>> 
>> >> >             ],
>> >> >             "name": "pc-i440fx-2.8",
>> >> >             "alias": "pc",
>> >> >             "is-default": true,
>> >> >             "cpu-max": 255,
>> >> >             "hotpluggable-cpus": true
>> >> >         },
>> >> >         [...]
>> >> >         {
>> >> >             "supported-device-types": [
>> >> >                 "sys-bus-device",
>> >> >                 "floppy",
>> >> >                 "isa-device",
>> >> >                 "ide-device"
>> >> >             ],
>> >> >             "cpu-max": 1,
>> >> >             "hotpluggable-cpus": true,
>> >> >             "name": "isapc"
>> >> >         },
>> >> >         [...]
>> >> >         {
>> >> >             "supported-device-types": [
>> >> >                 "sys-bus-device",
>> >> >                 "floppy",
>> >> >                 "i2c-slave",
>> >> >                 "pci-device",
>> >> >                 "isa-device",
>> >> >                 "ide-device"
>> >> >             ],
>> >> >             "cpu-max": 128,
>> >> >             "hotpluggable-cpus": true,
>> >> >             "name": "xenfv"
>> >> >         },
>> >> >         [...]
>> >> >         {
>> >> >             "alias": "q35",
>> >> >             "supported-device-types": [
>> >> >                 "sys-bus-device",
>> >> >                 "i2c-slave",
>> >> >                 "PCIE-device",
>> >> >                 "isa-device",
>> >> >                 "ide-device"
>> >> >             ],
>> >> >             "cpu-max": 288,
>> >> >             "hotpluggable-cpus": true,
>> >> >             "name": "pc-q35-2.8"
>> >> >         },
>> >> >         [...]
>> >> >     ]
>> >> >   }
>> >> > 
>> >> > Considered alternatives
>> >> > =======================
>> >> > 
>> >> > Indirect mapping (machine => bus => device)
>> >> > -------------------------------------------
>> >> > 
>> >> > This RFC implements a mechanism to implement ax
>> >> >   machine-type => supported-device-types
>> >> > mapping. An alternative solution I considered was to expose an
>> >> > indirect mapping:
>> >> >   machine-type => default-bus-types
>> >> > followed by
>> >> >   bus-type => supported-device-types.
>> >> > 
>> >> 
>> >> As I previously stated, I don't know if it helps. bus-type
>> >> can support different devices on different archs.
>> >
>> > Yes. After learning a bit more about PCIe, I am convinced that a
>> > bus->type => device-types mapping won't help.
>> >
>> >> 
>> >> > But exposing only the resulting supported device-types list
>> >> > imposes less restrictions on how the device and bus type
>> >> > hierarchy is implemented inside QEMU. There's still a
>> >> >   machine-type => bus-type => device-type
>> >> > mapping implemented internally, but it is an implementation
>> >> > detail on the current version, and not part of the
>> >> > externally-visible interface.
>> >> > 
>> >> 
>> >> Good, I personally don't like this "pass".
>> 
>> Point taken.
>> 
>> The core idea here seems to be using the QOM type hierarchy for
>> representing "pluggability".
>
> Exactly.
>
>> 
>> >> > The Implementation
>> >> > ==================
>> [Skipping for now...]
>> >> > 
>> >> > Limitations
>> >> > ===========
>> >> > 
>> >> > TYPE_SYS_BUS_DEVICE is too generic
>> >> > ----------------------------------
>> >> > 
>> >> > Currently all machines have a TYPE_SYS_BUS bus, meaning all
>> >> > TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
>> >> > 
>> >> 
>> >> Agreed, this is a problem.
>> >>
>> >> > The current solution in this series is to report
>> >> > TYPE_SYS_BUS_DEVICE as supported by all machines.
>> 
>> Because all machines have this catch-all pseudo-bus.  Reporting it
>> exists is correct, but as is the information is useless.
>
> I will probably keep returning it by default (to make the
> existing information correct), but provide mechanisms for
> machine-types to return more precise information.
>
>> 
>> >> >                                                   But we could
>> >> > gradually add arch-specific or machine-family-specific interface
>> >> > names that can be used on devices that are meant to work with
>> >> > only a subset of TYPE_SYS_BUS_DEVICE subclasses.
>> 
>> Basically, encode pluggability information in the QOM type hierarchy.
>> 
>> To gauge whether that would work I'd have to actually understand how
>> exactly pluggable sysbus devices work, in particular what the conditions
>> for accepting a plug are.
>
> I can't evaluate that, either.
>
> But note that covering 100% of the pluggability decisions for
> sysbus is not one of the problems I am trying to solve right now.
> I mean: if the new mechanisms can cover it too, great, but they
> don't (and there's no obvious solution for it), it is out of
> scope.

We don't have to solve all the problems right away.  But the less we
know about the problems we're not solving right away, the higher the
risk of accidentally creating obstacles to solving them later.

>> >> > A future version of this series may remove TYPE_SYS_BUS_DEVICE
>> >> > from the supported-device-types output, and return a
>> >> > arch-specific or machine-family-specific interface name to
>> >> > restrict management software to a subset of TYPE_SYS_BUS_DEVICE
>> >> > subclasses.
>> >> > 
>> >> > PCI vs PCIe
>> >> > -----------
>> >> > 
>> >> > Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on
>> >> > supported-device-types.
>> >> > 
>> >> > Machines with legacy PCI buses will report TYPE_PCI_DEVICE on
>> >> > supported-device-types.
>> >> > 
>> >> > The problem with the current approach is that PCIe devices are
>> >> > TYPE_PCI_DEVICE subclasses. The allows PCI device classes to
>> >> > indicate they are PCIe-capable, but there's no obvious way to
>> >> > indicate that a device is PCIe-only. This needs to be addressed
>> >> > in a future version of this series.
>> >> > 
>> >> > Suggestions are welcome.
>> >> 
>> >> 
>> >> As we talked offline, I personally like an interface IPCIType
>> >> with a field having 3 possible values {pci/pcie/hybrid}
>> >> 
>> >> To understand how hybrid works we need some rules, like
>> >> "pci on pci bus/pcie on pcie bus"
>> >> "pcie on a non-root pcie bus/pcie otherwise
>> >> 
>> >> I don't think we'll have a lot of rules, simple boolean fields
>> >> for the interface should be enough.
>> >
>> > What you propose makes sense, the only difference is that the
>> > boolean fields would be just interface names that can be used as
>> > the "implements" argument on qom-list-types.
>> >
>> > e.g.:
>> >
>> > * Hybrid PCI device-types would implement both "legacy-pci-device" and
>> >   "pcie-device" interfaces.
>> > * PCIe-only device-types would implement only the "pcie-device"
>> >   interface.
>> > * Legacy-PCI-only device-types would implement only the
>> >   "legacy-pci-device" interface.
>> 
>> In other words, encode "pluggability" in the QOM type hierarchy.
>> 
>> > Then, the bus instances would have a 'accepted-device-types'
>> > field:
>> >
>> > * A legacy PCI bus would accept only "legacy-pci-device" devices.
>> > * A PCIe-only bus would accept only "pcie-device" devices.
>> > * A PCIe bus that accepts legacy PCI devices (the root bus?)
>> >   would accept both "legacy-pci-device" and "pcie-device"
>> >   devices.
>> >
>> > Then, query-machines would return the list of bus instances that
>> > machine init creates, containing the bus ID, bus type, and
>> > accepted-device-types.
>> >
>> > Does that make sense?
>> 
>> Yes.
>> 
>> Our decision to have hybrid PCI/PCIe devices and buses breeds
>> considerable complexity.  I wish we had avoided them, but I believe it's
>> too late to change now.
>
> I'm diving into it right now. v2 will have some attempts to sort
> it out.
>
>> 
>> >> This still does not solve the problem that some devices makes
>> >> sense only on a specific arch.
>> 
>> Examples?
>> 
>> > Right now, we can simply compile out arch-specific devices that
>> > can never be plugged in a QEMU binary. But if we still want them
>> > compiled in for some reason, we can categorize them on a
>> > different type/interface name, and the corresponding machine-type
>> > would tell management that their buses accept only the
>> > arch-specific type/interface name.
>> 
>> Could become messy.  Can't say without looking at examples.
>
> Can't say, either.
>
>> 
>> >> > Incomplete bus lists on some machines
>> >> > -------------------------------------
>> >> > 
>> >> > With this series, not all machines classes are changed to add the
>> >> > full list of device types on the 'supported-device-types'. To
>> >> > allow the code to be updated gradually, qmp-machine-info.py has a
>> >> > STRICT_ARCHES variable, that will make the test code require a
>> >> > complete device type list only on some architectures.
>> 
>> Ideally, pluggability information would be derived from QOM.  I figure
>> that's the only practical way to solve the general case.
>> 
>> For the special case "newly created machine", deriving from QOM may be
>> impractical, say because you'd have to create the machine.  Then you
>> have to hard-code the result somehow.  Tests to validate the hardcoded
>> results match the results you get when you actually create the machine
>> would be required, at least once you *can* get results for a created
>> machine.
>
> Tests were implemented, see
>   [RFC 08/15] qmp: Add 'supported-device-types' field to 'query-machines'.
>
>> 
>> >> > Out of scope: Configurable buses
>> >> > --------------------------------
>> >> > 
>> >> > There's no way to map machine options like "usb=on|off" to
>> >> > device-types or buses. I plan to propose a new interface that
>> >> > allows machine options to be mapped to buses/device-types later.
>> >> > 
>> >> > Out of scope: Deciding where to plug devices
>> >> > --------------------------------------------
>> >> > 
>> >> > Once management software discovers which devices can be plugged
>> >> > to a machine, it still has to discover or define where devices
>> >> > can/should/will be plugged. This is out of the scope of this
>> >> > series.
>> >> > 
>> >> 
>> >> That's a pitty :(
>> >> I was hoping this series will solve this issue. But if it prepares
>> >> the grounds for it is also a good step .
>> >
>> > The bus ID will be in the scope of v2.
>> 
>> I'm afraid "bus" is insufficiently general.
>
> Does "socket" sound general enough?

Yes, because "socket" isn't defined, yet, so we can make it whatever it
needs to be :)

>> >> Thanks,
>> >> Marcel
>> >> 
>> >> > Out of scope: Hotplug
>> >> > ---------------------
>> >> > 
>> >> > The proposed interface is supposed to help management software
>> >> > find which device types can be used when creating the VM. Device
>> >> > hotplug is out of the scope of this series. However, the new
>> >> > 'device-types' QOM property on bus objects could be used to find
>> >> > out which device types can be plugged on the existing buses.