Re: [PATCH] make vfio and DAX cache work together

[Alex Williamson]

On Thu, 29 Apr 2021 09:44:51 +0100
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:

> * Alex Williamson (alex.williamson@redhat.com) wrote:
> > On Wed, 28 Apr 2021 20:17:23 +0100
> > "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> >   
> > > * Dev Audsin (dev.devaqemu@gmail.com) wrote:  
> > > > Thanks Dave for your explanation.
> > > > Any suggestions on how to make VFIO not attempt to map into the
> > > > unaccessible and unallocated RAM.    
> > > 
> > > I'm not sure;:
> > > 
> > > static bool vfio_listener_skipped_section(MemoryRegionSection *section)
> > > {
> > >     return (!memory_region_is_ram(section->mr) &&
> > >             !memory_region_is_iommu(section->mr)) ||
> > >            section->offset_within_address_space & (1ULL << 63);
> > > }
> > > 
> > > I'm declaring that region with memory_region_init_ram_ptr;  should I be?
> > > it's not quite like RAM.
> > > But then I *do* want a kvm slot for it, and I do want it to be accessed
> > > by mapping rather htan calling IO functions; that makes me think mr->ram
> > > has to be true.
> > > But then do we need to add another flag to memory-regions; if we do,
> > > what is it;
> > >    a) We don't want an 'is_virtio_fs' - it needs to be more generic
> > >    b) 'no_vfio' also feels wrong
> > > 
> > > Is perhaps 'not_lockable' the right thing to call it?  
> > 
> > This reasoning just seems to lead back to "it doesn't work, therefore
> > don't do it" rather than identifying the property of the region that
> > makes it safe not to map it for device DMA (assuming that's actually
> > the case).   
> 
> Yes, I'm struggling to get to what that generic form of that property
> is, possibly because I've not got an example of another case to compare
> it with.
> 
> > It's clearly "RAM" as far as QEMU is concerned given how
> > it's created, but does it actually appear in the VM as generic physical
> > RAM that the guest OS could program to the device as a DMA target?  If
> > not, what property makes that so, create a flag for that.  Thanks,  
> 
> The guest sees it as a PCI-bar; so it knows it's not 'generic physical
> RAM' - but can a guest set other BARs (like frame buffers or pmem) as
> DMA targets?  If so, how do I distinguish our bar?

They can, this is how peer-to-peer DMA between devices works.  However,
we can perhaps take advantage that drivers are generally a bit more
cautious in probing that this type of DMA works before relying on it,
and declare it with memory_region_init_ram_device_ptr() which vfio
would not consider fatal if it fails to map it.  The other semantic
difference is that ram_device_mem_ops are used for read/write access to
avoid some of the opcodes that are not meant to be used for physical
device memory with the default memcpy ops.  If you expect this region
to be mapped as a kvm memory slot, presumably these would never get
used anyway.  Thanks,

Alex