Re: [Qemu-devel] OVMF, Q35 and USB keyboard/mouse

[Laszlo Ersek]

On 09/15/14 17:07, Gabriel L. Somlo wrote:
> On Mon, Sep 15, 2014 at 05:01:21PM +0200, Laszlo Ersek wrote:
>>> diff --git a/hw/usb/hcd-ehci-pci.c b/hw/usb/hcd-ehci-pci.c
>>> index 289ca3b..bb230f1 100644
>>> --- a/hw/usb/hcd-ehci-pci.c
>>> +++ b/hw/usb/hcd-ehci-pci.c
>>> @@ -208,8 +208,8 @@ struct ehci_companions {
>>>  };
>>>  
>>>  static const struct ehci_companions ich9_1d[] = {
>>> -    { .name = "ich9-usb-uhci1", .func = 0, .port = 0 },
>>> -    { .name = "ich9-usb-uhci2", .func = 1, .port = 2 },
>>> +    { .name = "ich9-usb-uhci3", .func = 0, .port = 0 },
>>> +    { .name = "ich9-usb-uhci3", .func = 1, .port = 2 },
>>>      { .name = "ich9-usb-uhci3", .func = 2, .port = 4 },
>>>  };
>>>
>>>
>>> they *all* get detected and work great on ovmf+osx. Slow kbd+mouse
>>> get routed automatically to one of them, and work fine. The only
>>> differences I can see between them (in hw/usb/hcd-uhci.c) is
>>> the name string and "irq_pin" field. Not sure yet if that's likely
>>> to point to an explanation...
>>
>> It is actually extremely relevant, the irq_pin field. I'm not exactly
>> sure how just yet, but it is. Maybe check the interrupt routing in OSX
>> somehow? Do you have a dmesg-like log in OSX, with a PRT dump from the
>> DSDT, and messages about interrupt routing setup? Do you have in OSX
>> anything that corresponds to /proc/interrupts under Linux?
> 
> Actually, even more exciting:
> 
> diff --git a/hw/usb/hcd-uhci.c b/hw/usb/hcd-uhci.c
> index 3b3ebcd..d61656e 100644
> --- a/hw/usb/hcd-uhci.c
> +++ b/hw/usb/hcd-uhci.c
> @@ -1335,21 +1335,21 @@ static UHCIInfo uhci_info[] = {
>          .vendor_id = PCI_VENDOR_ID_INTEL,
>          .device_id = PCI_DEVICE_ID_INTEL_82801I_UHCI1,
>          .revision  = 0x03,
> -        .irq_pin   = 0,
> +        .irq_pin   = 1,
>          .unplug    = false,
>      },{
>          .name      = "ich9-usb-uhci2", /* 00:1d.1 */
>          .vendor_id = PCI_VENDOR_ID_INTEL,
>          .device_id = PCI_DEVICE_ID_INTEL_82801I_UHCI2,
>          .revision  = 0x03,
> -        .irq_pin   = 1,
> +        .irq_pin   = 2,
>          .unplug    = false,
>      },{
>          .name      = "ich9-usb-uhci3", /* 00:1d.2 */
>          .vendor_id = PCI_VENDOR_ID_INTEL,
>          .device_id = PCI_DEVICE_ID_INTEL_82801I_UHCI3,
>          .revision  = 0x03,
> -        .irq_pin   = 2,
> +        .irq_pin   = 3,
>          .unplug    = false,
>      },{
>          .name      = "ich9-usb-uhci4", /* 00:1a.0 */
> 
> Turns out, anything with an irq_pin <= 1 won't show up when osx is
> booted on q35 with ovmf (but osx + q35 works if booted via Chameleon).
> 
> DSDT looks identical across the ovmf vs. chameleon divide.

Now I'm curious. What's this chameleon thing? (Yes, I did find the
homepage. Apparently the lead developer is a fellow Hungarian. A small
world.) I'm surprised how you can get the same DSDT under both OVMF and
chameleon. Assuming you run a recent OVMF on a recent QEMU, the DSDT
exposed to the guest will originate from QEMU. This is confirmed by your
q35.log that you sent me in private (due to its size) previously:

InstallQemuLinkedTables: "etc/acpi/tables" offset 0x0000000000000000: 
Signature="FACS" Length=0x00000040
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x0000000000000040: 
Signature="DSDT" Length=0x00001CE5
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x0000000000001D25: 
Signature="FACP" Length=0x00000074
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x0000000000001D99: 
Signature="SSDT" Length=0x00000687
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x0000000000002420: 
Signature="APIC" Length=0x00000090
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x00000000000024B0: 
Signature="HPET" Length=0x00000038
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x00000000000024E8: 
Signature="QEMU" Length=0x0000003C
InstallQemuLinkedTables: "etc/acpi/tables" offset 0x0000000000002524: 
Signature="RSDT" Length=0x00000038
InstallAllQemuLinkedTables: installed 7 tables

I've got no clue how you can end up with the exact same DSDT under
chameleon, unless it has a client for QEMU's fw_cfg ACPI linker/loader.

In fact I think that should be *precisely* the difference here. The PCI
interrupt routing table (_PRT) in the DSDT describes a two-level
mapping. (I've probably forgotten most of the details, sorry.) First, it
maps each PCI (bus, dev, pin) triplet to a PNP0C0F ("PCI interrupt
link") device. We usually call these LNKA, LNKB, LNC, LNKD, LNKS on
i440fx; there are more on q35. Then, each LNKx specifies a set of
possible legacy interrupts that the link can be programmed for /
assigned to. At runtime, the OS programs each of the interrupt links to
one of its allowed legacy interrupts, and then all the pins (across
buses and functions) that are connected to that interrupt link will
trigger that interrupt. The OS usually tries to come up with a mapping
(from LNKx to IRQ) so that interrupt sharing is minimized.

Here's an example from my i440fx Fedora 20 VM.

(1) The dmesg says first

  ACPI: PCI Interrupt Link [LNKA] (IRQs 5 10 *11)
  ACPI: PCI Interrupt Link [LNKB] (IRQs 5 10 *11)
  ACPI: PCI Interrupt Link [LNKC] (IRQs 5 *10 11)
  ACPI: PCI Interrupt Link [LNKD] (IRQs 5 *10 11)
  ACPI: PCI Interrupt Link [LNKS] (IRQs *9)

This displays what IRQs the _PRT in the DSDT allows for each of the LNKx
links, and the asterisks show (IIRC) what elements of those sets are
selected (programmed) when Linux inherits the hardware.

(2) Later it logs

  ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
  ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
  ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11
  ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10

Let's call this mapping LNK_IRQ().

(3) Then look for the uchi controllers:

  uhci_hcd 0000:00:07.0: irq 10, io base 0x0000c0c0
  uhci_hcd 0000:00:07.1: irq 11, io base 0x0000c0a0
  uhci_hcd 0000:00:07.2: irq 11, io base 0x0000c080

And /proc/interrupts is consistent with that:

             CPU0       CPU1       
   10:          6         25   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb2
   11:          0          0   IO-APIC-fasteoi   uhci_hcd:usb3, uhci_hcd:usb4, 
virtio2

These last two blocks are *results*.

Again, this is the result of composing two functions:

  device_interrupt = LNK_IRQ(PRT(bus, dev, pin))

PRT() comes from the DSDT, and maps (bus, dev, pin) to a link, while
LNK_IRQ() comes from the OS (the actual link -> IRQ assignment), and is
restricted to the possibilities offered in the DSDT.

The PRT that QEMU generates follows a rotating pattern (it is not
restricted by physical circuits). As you go from one PCI device to the
next, the same LNKA - LNKD links are distributed over the device's pins,
but the sequence is shifted by one. The idea is that most PCI devices
use only their first pin (INTA), and placing such devices "beside" each
other should nicely iterate over all links, evenly.

Thus far I didn't speak about functions of the same PCI device. I didn't
do that because I'm uneducated (even more than in the above :)). The
basic idea is that different functions of the device will use different
pins. Most devices are single-function, hence they usually stick with
INTA. If you've got a multifunction device, then the functions will use
separate pins.

For example, if I dump and decompile the DSDT in the guest, for bus 0
device 7, I get, from the PRT:

  Package (0x04) { 0x0007FFFF,  Zero,  LNKC,  Zero },  <-- pin 0 / INTA
  Package (0x04) { 0x0007FFFF,  One,  LNKD,  Zero },   <-- pin 1 / INTB
  Package (0x04) { 0x0007FFFF,  0x02,  LNKA,  Zero },  <-- pin 2 / INTC
  Package (0x04) { 0x0007FFFF,  0x03,  LNKB,  Zero },  <-- pin 3 / INTD

Let's put it all together, for a QEMU command line with

  -device 
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7
  -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1
  -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2

dev & func,  pin, set  interrupt link  IRQ programmed
set on qemu  in qemu   for dev & pin,  by Linux for
cmdline      source    set in DSDT     link
-----------  --------  --------------  --------------
07.0                0            LNKC              10
07.1                1            LNKD              11
07.2                2            LNKA              11

The first two columns are input parameters (from the command line and
from the source code). The third colum is the result of evaluating PRT()
on the input params. The fourth column is the result of evaluating
Linux's LNK_IRQ() function on the third column.

(Note that all of the above is for i440fx, not q35, but the method is
similar.)

Ultimately, I think that the difference between OVMF and chameleon is
the following: when booting OSX with chameleon, QEMU's rotating _PRT is
not exposed to OSX, because chameleon doesn't know how to download and
interpret the necessary fw_cfg blobs. (What _PRT OSX decides to use
then, I can't imagine.) But when you boot OSX with OVMF, then QEMU's
_PRT is exposed to OSX, and OSX, seeing the PCI bus/device/func
addresses of the UHCI controllers, *and* seeing their respective PINs,
*and* seeing their respective LNKx links (from the DSDT), maps the
function to some interrupt that kills the device.

This is consistent with your results (if you change the PINs in the
source code, things work). It would be interesting to see what happens
if you shuffle the PCI addresses of the UHCI controllers.

... Hm. You did mention in the thread starter that chameleon runs on top
of SeaBIOS. SeaBIOS does have an ACPI linker/loader client, which would
explain why you see the same DSDT. The only thing that could differ
between the two cases is the LNK_IRQ() assignment then (ie. how OSX
chooses to map PCI interrupt links to IRQs), and I don't know why that
would be different. A /proc/interrupts table would be useful, again.

(Sorry about all the crazy errors I must have said above about PCI, ACPI
etc etc etc. Even if it turns out to be incorrect to some degree, if it
helps others help you, then it wasn't in vain.)

Laszlo

> I'm going
> to take some time to do a more thorough search of any and all logging
> I can find on OS X, and also try to find wherever in ovmf the guest
> device irq pin comes into play where uhci is concerned.
> 
> I am basically just poking at it with a stick right now, but maybe
> something will start making sense soon :)
> 
> Thanks again,
> --Gabriel
>