Re: [Qemu-devel] bypassing pci_dma_read() and pci_dma_write() ?

[Paolo Bonzini]

Il 18/01/2013 17:04, Luigi Rizzo ha scritto:
> Hi,
> with a bunch of e1000 improvements we are at a point where we are
> doing over 1Mpps (short frames) and 7-8Gbit/s (1500 byte frames)
> between two guests, and two things that are high in the "perf top"
> stats are phys_page_find() and related memory copies.
> 
> Both are triggered by the pci_dma_read() and pci_dma_write(),
> which on e1000 (and presumably other frontends) are called on
> every single descriptor and every single buffer.
> 
> I have then tried to access the guest memory without going every
> time through the page lookup. [...]
> 
> This relies on the assumption that the ring (which is contiguous in the
> guest's physical address space) is also contiguous in the host's virtual
> address space.  In principle the property could be easily verified once
> the ring is set up.

IIRC, the amount of contiguous memory is written by address_space_map in
the plen parameter.

In your case:

>     +       s->txring = address_space_map(pci_dma_context(&s->dev)->as,
>     +                       base, &desclen, 0 /* is_write */);

that would be desclen on return from address_space_map.

> And of course, am i missing some important detail ?

Unfortunately yes.

First, host memory mappings could change (though they rarely do on PC).
 The result of address_space_map is not guaranteed to be stable.  To
avoid problems with this, however, you could use something like
hw/dataplane/hostmem.c and even avoid address_space_map altogether.

Second, that pci_dma_*() could have the addresses translated by an
IOMMU.  virtio is documented to have "real" physical memory addresses,
but this does not apply to other devices.

Paolo

> Of course the above could be used conditionally if the required
> conditions hold, and then revert to the pci_dma_*()
> in other cases.
> 
> cheers
> luigi
> 
>