Re: [Qemu-devel] Default for phys-addr-bits? (was Re: [PATCH 4/5] x86: Allow physical address bits to be set)

[Dr. David Alan Gilbert]

* Gerd Hoffmann (address@hidden) wrote:
> > So that's mapped at an address beyond host phys-bits.
> > And it hasn't failed/crashed etc - but I guess maybe nothing is using that 
> > 2G space?
> 
> address@hidden ~# dmesg | grep Surface
> [    4.830095] [drm] qxl: 2048M of Surface memory size
> 
> qxl bar 4 (64bit) and qxl bar 1 (32bit) are the same thing.  The 64bit
> bar can be alot larger obviously.  The 32bit bar is just an alias for
> the first portion of the 64bit bar.  So I guess qxl just falls back to
> use bar 1 instead of bar 4 because ioremap() on bar 4 fails.
> 
> > Obviously 128T is a bit silly for maxmem at the moment, however I was 
> > worrying what
> > happens with 36/39/40bit hosts, and it's not unusual to pick a maxmem 
> > that's a few TB
> > even if the VMs you're initially creating are only a handful of GB. 
> > (oVirt/RHEV seems to use
> > a 4TB default for maxmem).
> 
> Oh, ok.  Should be fixed I guess.
> 
> > Still, this only hits as a problem if you hit the combination of:
> >    a) You use large PCI bars
> 
> ovmf will map all 64bit bars high, even without running out of 32bit
> address space.  And with virtio 1.0 pretty much every virtual machine
> will have 64bit bars.

OK, yes I got that working, and you're right, it does map it high;
(with recent OVMF running virtio 1.0, that needed recent guest/host kernels)
and it does fail easily as well if memory doesn't fit, so for
example:

(all on a xeon with 46 bit physical host)

specifying maxmem=1T - upstream build is hanging - but works if I specify 
phys-bits=46
    so yes, it's noticing if the guest phys-bits is too small
    even if the host can manage it.

It's OK if running with small amount of RAM and phys-bits=40

maxmem=64T with any phys-bits hangs.


specifying maxmem=64T with phys-bits=46 on xeon and it hangs
specifying maxmem=64T with phys-bits=48 on xeon and it hangs
specifying maxmem=32T with phys-bits=46-48 on xeon and it works

So for example we see:
  Bus  2, device   4, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 10.
      BAR1: 32 bit memory at 0x98000000 [0x98000fff].
      BAR4: 64 bit prefetchable memory at 0x200800000000 [0x2008007fffff].
      id "virtio-disk0"

  and that works nicely.

Dave

> 
> cheers,
>   Gerd
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK