Re: [Qemu-devel] [RFC] edk2 support for a new QEMU device - PXB (PCI Expander Device)

[Laszlo Ersek]

On 06/03/15 22:34, Marcel Apfelbaum wrote:
> On 06/03/2015 01:20 PM, Laszlo Ersek wrote:

>> Maybe we can experiment some more; for example we could start by
>> you explaining to me how exactly to probe for a root bus's presence
>> (you mentioned device 0, but I'll need more than that).

> Well, I lied. :)
> I had a look now on seabios and it does the following:
> - Receives using a fw_config file the number of extra root buses.
> - It starts scanning from bus 0 to bus 0xff until it discovers all
>   the extra root buses. The 'discovery' is "go over all bus's slots
>   and probe for a non empty PCI header". If you find at least one
>   device you just discovered a new PCI root bus.

I thought about checking the VendorId header field for dev=0 func=0 on
each bus. (Sources on the net indicate that the VendorId field is
usually queried for presence -- all bits one means "nope".)

> I think that we can improve the fw_config file to pass the actually
> bus numbers and not only the total. In this way should be relatively
> easy for edk2 to handle the extra root buses.

Yes. I had thought this would be the easiest. I wasn't sure though if
you'd appreciate such an idea :)

>> For the bus range allocation, here's an idea:
>> - create a bitmap with 256 bits (32 bytes) with all bits zero
>> - probe all root buses; whatever is found, flip its bit to 1
>> - assuming N root buses were found, divide the number of remaining
>>   zero bits with N. The quotient Q means how many subordinate buses
>>   each root bus would be able to accommodate
>> - for each root bus:
>>    - create an ACPI bus range descriptor that includes only the root
>>      bus's number
>>    - pull out Q zero bits from the bitmap, from the left, flipping
>>      them to one as you proceed
>>    - for each zero bit pulled, try to append that bus number to the
>>      ACPI bus range descriptor (simply bumping the end). If there's a
>>      discontinuity, start a new ACPI bus range descriptor.
>>
>> This greedy algorithm would grant each root bus the same number of
>> possible subordinate buses, could be implemented in linear time, and
>> would keep the individual bus ranges "reasonably continuous" (ie.
>> there should be a reasonably low number of ACPI bus range
>> descriptors, per root bus).
>>
>> What do you think? This wouldn't be a very hard patch to write, and
>> then we could experiment with various -device pxb,bus_nr=xxx
>> parameters.

> Well, it looks nice but I think that we can do something much simpler
> :)
> Let's continue the above idea that QEMU passes to edk2 the *extra*
> root bus numbers in ascending order for simplicity.
> For example 8,16,32. From here you can derive that the bus ranges are:
> 0-7 host bridge 0
> 8-15 pxb root bridge 1
> 16-31 pxb root bridge 2
> 32-0xff pxb root bridge 3

Sounds good, at least if the bus numbers assigned to the pxb's partition
the full range fairly uniformly.

> BTW, this is the way, as far as I know, that the real hw divides the
> ranges.
> Limitation:
>   - How do you know you have enough bus numbers for a host bridge to
>     cover all PCI-2-PCI bridges behind it? Let's say bus 0 has 10
>     bridges, 0-7 range is not enough.

Exactly.

> Reasoning:
>   - This is *hw vendor* issue, not firmware, in our case QEMU should
>     check the ranges are enough before starting edk2.

If you're willing to do the work in QEMU, you certainly won't meet any
resistance on my part! :)

> In conclusion, this assumption does not break anything or gives as a
> big limitation.
> And Seabios already assumes that... and QEMU is not going to break it.

Great!

>> The MMIO and IO spaces I would just share between all of them; the
>> allocations from those are delegated back to the host bridge / root
>> bridge driver, and the current implementation seems sufficient -- it
>> just assings blocks from the same big MMIO ( / IO) space downwards

> Yes, this is how it should be done, I am happy that it already works
> that way.

Tonight I've started to work on this anyway. Before attacking the bitmap
idea, I wanted to -- had to, really -- rewrap OVMF's fresh clone of
"PcAtChipsetPkg/PciHostBridgeDxe" to 79 columns. I expect to delve into
the driver more deeply this time than last time, and the consistently
overlong (130-148 character) lines make the code simply unreadable.

So, I just finished that. (It was surprisingly difficult; the rewrapping
took 8 patches, the cumulative diffstat is 9 files changed, 2261
insertions(+), 1445 deletions(-).) I thought I'd check my email before
embarking on the bitmap thing. Your email arrived at the best possible
moment! Not just because I don't have to implement the bitmap, the
search, the multiple ACPI bus ranges per root bridge, but also because
the internals of the driver rely quite heavily on each root bridge
having a single contiguous bus range.

I think I could have rebased that to bitmap checks, but the approach
you're suggesting makes it all unnecessary. (Plus, I don't have to worry
about any incompatibility with the PCI bus driver, which I won't touch.)

What element type do you propose for the array in the new fw_cfg file?
(And what name for the fw_cfg file itself?)

"etc/extra-pci-roots" uses uint64_t, little endian, for the number of
extra root buses. (In fact if you expose the explicit list in a separate
file, then the element count is not even necessary separately, because
file sizes are available in the fw_cfg directory, and I can divide the
file size with the element size.)

I have two more questions (raised earlier), about the _HID and the _UIDs
in the SSDT.

First, I can see in your patch

  hw/acpi: add support for i440fx 'snooping' root busses

that the _UID is populated for each root bus with a string of the form

  PC%02X

where the argument is "bus_num". UEFI can accommodate this, with the
Expanded ACPI Device Path node, but I'll have to know if the "bus_num"
argument matches the exact numer that you're going to pass down in the
new fw_cfg file. Does it?

Second, about the _HID and _UID objects being populated by strings, and
not numbers. In *theory* this should be all fine, but I'm concerned that
in practice they will cause complications (eg. in booting).

Can you perhaps change (or update, separately) the QEMU patch in
question, so that _HID is populated with aml_eisaid() instead of
aml_string(), and that _UID is populated with a flat integer (the
"bus_num" value)? It should not be very intrusive for QEMU at this
point, and down the road I think it would ensure better compatibility.
Something to the tune of:

> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index db32fd1..8fae3b9 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -944,9 +944,8 @@ build_ssdt(GArray *table_data, GArray *linker,
>
>              scope = aml_scope("\\_SB");
>              dev = aml_device("PC%.02X", bus_num);
> -            aml_append(dev,
> -                       aml_name_decl("_UID", aml_string("PC%.02X", 
> bus_num)));
> -            aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
> +            aml_append(dev, aml_name_decl("_UID", aml_int(bus_num)));
> +            aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A03")));
>              aml_append(dev, aml_name_decl("_BBN", aml_int(bus_num)));
>
>              if (numa_node != NUMA_NODE_UNASSIGNED) {

As far as I can see in the QEMU source, filling in _HID and _UID like
this is existing practice.

Thanks!
Laszlo