Re: RFC: use VFIO over a UNIX domain socket to implement device offloading

[John G Johnson]

> On Jun 23, 2020, at 5:27 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> 
> On Thu, Jun 18, 2020 at 02:38:04PM -0700, John G Johnson wrote:
>>> On Jun 15, 2020, at 3:49 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>>> An issue with file descriptor passing is that it's hard to revoke access
>>> once the file descriptor has been passed. memfd supports sealing with
>>> fnctl(F_ADD_SEALS) it doesn't revoke mmap(MAP_WRITE) on other processes.
>>> 
>>> Memory Protection Keys don't seem to be useful here either and their
>>> availability is limited (see pkeys(7)).
>>> 
>>> One crazy idea is to use KVM as a sandbox for running the device and let
>>> the vIOMMU control the page tables instead of the device (guest). That
>>> way the hardware MMU provides memory translation, but I think this is
>>> impractical because the guest environment is too different from the
>>> Linux userspace environment.
>>> 
>>> As a starting point adding DMA_READ/DMA_WRITE messages would provide the
>>> functionality and security. Unfortunately it makes DMA expensive and
>>> performance will suffer.
>>> 
>> 
>>      Are you advocating for only using VFIO_USER_DMA_READ/WRITE and
>> not passing FDs at all?  The performance penalty would be large for the
>> cases where the client and server are equally trusted.  Or are you
>> advocating for an option where the slower methods are used for cases
>> where the server is less trusted?
> 
> I think the enforcing IOMMU should be optional (due to the performance
> overhead) but part of the spec from the start.
> 


        With this in mind, we will collapse the current memory region
messages (VFIO_USER_ADD_MEMORY_REGION and VFIO_USER_SUB_MEMORY_REGION)
and the IOMMU messages (VFIO_USER_IOMMU_MAP and VFIO_USER_IOMMU_UNMAP)
into new messages (VFIO_USER_DMA_MAP and VFIO_USER_DMA_UNMAP).  Their
contents will be the same as the memory region messages.

        On a system without an IOMMU, the new messages will be used to
export the system physical address space as DMA addresses.  On a system
with an IOMMU they will be used to export the valid device DMA ranges
programmed into the IOMMU by the guest.  This behavior matches how the
existing QEMU VFIO object programs the host IOMMU.  The server will not
be aware of whether the client is using an IOMMU.

        In the QEMU VFIO implementation, will will add a ‘secure-dma’
option that suppresses exporting mmap()able FDs to the server.  All
DMA will use the slow path to be validated by the client before accessing
guest memory.

        Is this acceptable to you (and Alex, of course)?

                                                                JJ