General driver DMA (in EROS)

[Jonathan S. Shapiro]

Disk DMA is a special case because the objects (e.g. the pages) don't
really exist until they are loaded, so there is nothing to DMA into.

For the more general case, there are three approaches that we have
taken, depending on what the problem space is. There are two scenarios:

  1. You are doing requested I/O -- that is, the user has initiated
     some I/O and you are returning bytes. The key issue here is that
     no (de)multiplexing of data is occurring on the channel at I/O
     time.

  2. You are doing unsolicited I/O -- e.g. packets are coming in and
     you need to demultiplex them.


So there are really three cases:

1. Solicited I/O with extra copy

   Driver simply does I/O into driver-reserved memory, and returns it
   via the IPC protocol as a copied string. Appropriate for small data
   on low-frequency devices. Probably acceptable for terminal lines.

2. Solicited, zero copy I/O

   Requester provides a capability to an address space region. Driver
   uses a high-privilege capability to discover the corresponding
   physical page addresses and pins the frames for I/O. Unpins them
   when done.

     [Aside: attempts to destroy an object that is pinned for I/O
     will block until the I/O is completed.]

3. Unsolicited I/O

   In our experience, the driver is going to need to make a temporary
   copy long enough to demux the incoming data. For example, the
   ethernet driver needs to run an incoming packet filter to determine
   the recipient.

   What we have done here is do the incoming into driver-allocated
   memory, and do a copy-out into client-provided memory as the
   packet is demultiplexed.

   It is unclear what the right answer will be when it becomes possible
   to download the filtering onto the ethernet board. At that point we
   would probably want to fall back to the I/O pinned model.

Note that in order for I/O pin to be reasonable, we need to have a bound
on the total number of frames involved in this situation.


A final possibility that we have considered but never tried. It arises
because of the desire to use unmodified Linux drivers.

The problem with linux drivers is that they only work most of the time.
Some are very good. Most are simply lucky. We really want to sandbox
these things, not because we think they are hostile, but simply because
we cannot afford an undetected oops.

One piece of technology we could use to do this is dynamic translation.
My lab has a very fast dynamic translator that we could use to implement
an I/O MMU in software. This tool could also manage the I/O pin logic
transparently.

Actually, I was supposed to publish the tarball last week on our web
site. Instead, I wrote several megabytes of email. :-)

shap