Re: [Qemu-devel] SeaBIOS, FW_CFG_NUMA, and FW_CFG_MAX_CPUS

[Gleb Natapov]

On Fri, Jul 20, 2012 at 05:00:25PM -0300, Eduardo Habkost wrote:
> Hi,
> 
> While working at the CPU index vs APIC ID changes, I stumbled upon
> another not-very-well-defined interface between SeaBIOS and QEMU, and I
> would like to clarify the semantics and constraints of some FW_CFG
> entries.
> 
> First, the facts/assumptions:
> 
> - There's no concept of "CPU index" or "CPU identifier" that SeaBIOS and
>   QEMU agree upon, except for the APIC ID. All SeaBIOS can really see
>   are the CPU APIC IDs, on boot or on CPU hotplug.
> - The APIC ID is already a perfectly good CPU identifier, that is
>   present on bare metal hardware too.
>   - Adding a new kind of "CPU identifier" in addition to the APIC ID
>     would just make things more complex. 
>   - The only problem with APIC IDs is that they may not be contiguous.
> 
> That said, I would like to clarify the meaning of:
> 
> - FW_CFG_MAX_CPUS
> 
> What are the basic semantics and expectations about FW_CFG_MAX_CPUS?
> Considering that the APIC IDs may not be contiguous, is it supposed to
> be:
> 
> a) the maximum number of CPUs that will be ever online, doesn't matter
>    their APIC IDs, or
> b) a value so that every CPU has APIC ID < MAX_CPUS.
> 
> A practical example: suppose we have a machine with 18 CPUs with the
> following APIC IDs: 0x00, 0x01, 0x02, 0x04, 0x05, 0x06, 0x08, 0x09,
> 0x0a, 0x10, 0x11, 0x12, 0x14, 0x15, 0x16, 0x18, 0x19, 0x1a.
> 
> (That's the expected result for a machine with 2 sockets, 3 cores per
> socket, 3 threads per core.)
> 
> In that case, should FW_CFG_MAX_CPUS be: a) 18, or b) 27 (0x1b)?
> 
> If it should be 18, it will require additional work on SeaBIOS to make:
> - CPU hotplug work
> - SRAT/MADT/SSDT tables be built with Processor ID != APIC ID
> - SRAT/MADT/SSDT tables be kept stable if the system is hibernated and
>   resumed after a CPU is hot-plugged.
> 
> (Probably in that case I would suggest introducing a FW_CFG_MAX_APIC_ID
> entry, so that SeaBIOS can still build the ACPI tables more easily).
> 
> 
> - FW_CFG_NUMA
> 
> The first problem with FW_CFG_NUMA is that it depends on FW_CFG_MAX_CPUS
> (so it inherits the same questions above). The second is that
> FW_CFG_NUMA is a CPU-based table, but there's nothing SeaBIOS can use to
> know what CPUs FW_CFG_NUMA is refering to, except for the APIC IDs. So,
> should FW_CFG_NUMA be indexed by APIC IDs?
> 
> 
> - My proposal:
> 
> My proposal is to try to keep things simple, and just use the following
> rule:
> 
>  - Never have a CPU with APIC ID > FW_CFG_MAX_CPUS.
> 
> This way:
> - The SeaBIOS ACPI code can be kept simple.
> - The current CPU hotplug interface can work as-is (up to 256 CPUs),
>   based on APIC IDs.
> - The current FW_CFG_NUMA interface can work as-is, indexed by APIC IDs.
> - The ACPI tables can be easily kept stable between hibernate and
>   resume, after CPU hotplug.
> 
> This is the direction I am trying to go, and I am sending this just to
> make sure nobody is against it, and to not surprise anybody when I send
> a QEMU patch to make FW_CFG_MAX_CPUS be based on APIC IDs.
> 
This shouldn't change the meaning of maxcpus on command line though.
Qemu can calculate max ACPI ID needed to support maxcpus by itself.

> 
> My second proposal would be to introduce a FW_CFG_MAX_APIC_ID entry, so
> the SeaBIOS ACPI code can be kept simple.
> 
> My third proposal would be to introduce a FW_CFG CPU Index => APIC ID
> table, but I really wouldn't like to introduce a new type of CPU
> identifier to be used between QEMU and SeaBIOS, when the APIC ID is a
> perfectly good unique CPU identifier that already exists in bare metal
> hardware.
> 
> -- 
> Eduardo

--
                        Gleb.