Re: [Qemu-devel] Re: [PATCH 2 of 5] add can_dma/post_dma for direct IO

[Blue Swirl]

On 12/16/08, Anthony Liguori <address@hidden> wrote:
> Avi Kivity wrote:
>
> > Blue Swirl wrote:
> >
> > >
> > > >  I don't understand.  It's not a device that needs bouncing, it's a
> > > > particular transfer.  This could be either due to the transfer
> targeting
> > > > mmio, or due to the transfer requiring a transformation.
> > > >
> > > >
> > >
> > > Should the bouncing be something more much complex, for example
> > > negotiated between the devices? Or maybe the devices set up a
> > > transforming and non-transforming channel (which the other end should
> > > be able to transform some more) and use them as needed?
> > >
> > >
> >
> > Yes.  We already have two cases:
> >
> > - may do partial request: useful for block storage where requests can be
> huge
> > - need full request: networking, where you can't send or receive half a
> packet; on the other hand, packets are small (even with tso)
> >
> > You're adding a third case, always bounce, when the data undergoes a
> transformation which can't happen in-place.
> >
>
>  IMHO, IOMMU translation is distinct from mapping/data transformation.  I
> would think the IOMMU translation API would be different in that it took a
> physical address and returned a physical address.

Fully agree. On Sparc32, the incoming address is called DVMA (device
virtual memory address).

>  The IOMMU DMA API (which could transform data potentially) should return a
> virtual address and data a physical address.  In the normal case
> (non-transforming IOMMU), it should just fall-through to CPU DMA API (which
> is just map/unmap).
>
>  The PCI DMA API would normally fall through to the IOMMU DMA API.
>
>  So we would have:
>
>  PCI DMA map(addr):
>   if IOMMU:
>     return IOMMU DMA map(addr)
>   return CPU DMA map(addr)
>
>  IOMMU DMA map(addr):
>   new_address = IOMMU translate(addr)
>   if transform:
>    return IOMMU byte-swap map(addr)
>   return CPU DMA map(addr)

But with this we would be back to the original merged API.

I'd propose three to four distinct steps:
1. Resolve the device addresses via IOMMU etc. to guest physical
addresses (typically no changes if no IOMMU)
2. Negotiate bouncing (if bounced, the step returns a host address, if
not, a physical address)
3. Map the physical addresses to host addresses, bypass for the bounce cases
4. Perform vectorized zero-copy AIO. (Profit?)

2 & 3 may/should be merged but I'm not sure we have the full picture yet.