Re: [Qemu-devel] [Bug 1769053] Re: Cannot start a guest with more than 1TB of RAM

[Gerd Hoffmann]

  Hi,

>   d) For smaller amount of RAM it might still fail if
> RAM+rounding+pci+hotplug space goes over the limit.
>      Figuring that limit out is tricky (and I thought it
>      might be BIOS/EFI dependent as well depending where they
>      decide to put their PCI devices)

Both seabios and ovmf try to not go too high in address space.  Reason
is exactly the phys-bits issue.  Using 40 here by default does not only
limit the memory to 1TB.  It also has the problem that the guest thinks
it has 1TB of address space but in reality it might be less.  Even
recent skylake machines have phys-bits=39 (512G) only, and trying to use
the physical address space above 512G in the guest just doesn't work
because the phys-bits=39 limit applies to EPT too.

So checking phys-bits in the firmware, for example to place pci bars as
high as possible in physical address space, is not going to work.

IIRC ovmf uses a 32G sized region with 32G alignment by default, which
will land below 64G (aka phys-bits=36 address space) unless the guest
has more than 30 (q35) or 31 (piix4) GB of memory.

seabios will not map pci bars above 4G unless it runs out of space below
4G.  If needed 64bit PCI bars will be placed right above ram, with
gigabyte alignment.

cheers,
  Gerd