Re: [Qemu-devel] [RFC/RFT PATCH v2 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Michael S. Tsirkin]

On Thu, May 14, 2015 at 04:19:23PM +0200, Laszlo Ersek wrote:
> On 05/14/15 15:48, Michael S. Tsirkin wrote:
> > On Thu, May 14, 2015 at 03:32:10PM +0200, Laszlo Ersek wrote:
> >> On 05/14/15 15:00, Andrew Jones wrote:
> >>> On Thu, May 14, 2015 at 01:38:11PM +0100, Peter Maydell wrote:
> >>>> On 14 May 2015 at 13:28, Paolo Bonzini <address@hidden> wrote:
> >>>>> Well, PCI BARs are generally MMIO resources, and hence should not be 
> >>>>> cached.
> >>>>>
> >>>>> As an optimization, OS drivers can mark them as cacheable or
> >>>>> write-combining or something like that, but in general it's a safe
> >>>>> default to leave them uncached---one would think.
> >>>>
> >>>> Isn't this handled by the OS mapping them in the 'prefetchable'
> >>>> MMIO window rather than the 'non-prefetchable' one? (QEMU's
> >>>> generic-PCIe device doesn't yet support the prefetchable window.)
> >>>
> >>> I was thinking (with my limited PCI knowledge) the same thing, and
> >>> was planning on experimenting with that.
> >>
> >> This could be supported in UEFI as well, with the following steps:
> >> - the DTB that QEMU provides UEFI with should advertise such a
> >>   prefetchable window.
> >> - The driver in UEFI that parses the DTB should understand that DTB
> >>   node (well, record type), and store the appropriate base & size into
> >>   some new dynamic PCDs (= basically, firmware wide global variables;
> >>   PCD = platform configuration database)
> >> - The entry point of the host bridge driver would call
> >>   gDS->AddMemorySpace() twice, separately for the two different windows,
> >>   with their appropriate caching attributes.
> >> - The host bridge driver needs to be extended so that TypePMem32
> >>   requests are not rejected (like now); they should be handled
> >>   similarly to TypeMem32. Except, the gDS->AllocateMemorySpace() call
> >>   should allocate from the prefetchable range (determined by the new
> >>   PCDs above).
> >> - QEMU's emulated devices should then expose their BARs as prefetchable
> >>   (so that the above branch would be taken in the host bridge driver).
> >>
> >> (Of course, if QEMU intends to emulate PCI devices somewhat
> >> realistically, then QEMU should claim "non-prefetchable" for BARs that
> >> would not be prefetchable on physical hardware either, and then the
> >> hypervisor should accommodate the firmware's UC mapping and say "hey I
> >> know better, we're virtual in fact", and override the attribute (-> use
> >> WB instead of UC). With which we'd be back to square one...)
> >>
> >> Thanks
> >> Laszlo
> > 
> > Prefetcheable is unrelated to BAR caching or drivers, it's a way to tell
> > host bridges they can do limited tweaks to downstream transactions in a
> > specific range.
> > 
> > Really non-prefetcheable BARs are mostly those where read has
> > side-effects, which is best avoided. this does not mean it's ok to
> > reorder transactions or cache them.
> 
> I believe I understood that (although certainly not in the depth that
> you do), because when the idea had come up first (ie. equating cacheable
> with prefetchable, or at least "repurposing" the latter for the former)
> I had tried to read up on prefetchable (just on the web; no time for
> reading the PCI spec. ... I peeked now, it also mentions "write merging"
> for bridges.)

Read up on what it is if you like, it is much weaker than WC not to
mention cacheable.

> The way I perceived it, the idea was to give the guest a
> hint about caching with the prefetchable bit / DTB entry. Sorry if I was
> mistaken.
> 
> Thanks
> Laszlo

And what I am saying is that prefetchable bit would be a PV solution -
on real devices it is not a hint about caching and can't be used as
such.

-- 
MST