Re: [Qemu-devel] device assignment for embedded Power

[Scott Wood]

On Fri, 1 Jul 2011 21:59:35 +0100
Paul Brook <address@hidden> wrote:

> > On Fri, 1 Jul 2011 18:03:01 +0100
> > 
> > Paul Brook <address@hidden> wrote:
> > > Basically you should start by implementing full emulation of a device
> > > with similar characteristics to the one you want to passthrough.
> > 
> > That's not going to happen.
> 
> Why is your device so unique? How does it interact with the guest system and 
> what features does it require that doen't exist in any device that can be 
> emulated?

Perhaps I misunderstood what you meant by "similar characteristics".  I see
no reason to spend a bunch of time implementing full emulation for a device,
that isn't going to be used, just because it seems like a nice
intermediary step.

What specifically is it you're suggesting we do full emulation of?

> I'm also extremely sceptical of anything that only works in a kvm 
> environment.  
> Makes me think it's an unmaintainable hack, and almost certainly going to 
> cause you immense amounts of pain later.

I believe the only part of the device assignment stuff we've implemented so
far that is KVM specific is the interrupt routing.  I'm open to ways of
routing the interrupts to qemu in the non-KVM case, as long as we can
bypass it when KVM is used.

I'm not sure what the use case is for direct assignment of a device in an
otherwise completely emulated guest, but perhaps there is one.

> > > I doubt you're going to get generic passthrough of arbitrary devices
> > > working in a useful way.
> > 
> > It's usefully working for us internally -- we're just trying to find a way
> > to improve it for upstream, with a better configuration mechanism.
> 
> I don't believe that either.  More likely you've got passthrough of device 
> hanging off your specific CPU bus, using only (or even a subset of) the 
> facilities provided by that bus.

There's nothing special about our "bus".  It's MMIO, DMA, and interrupts.

What specifically are you disbelieving?

> > > Basically you have to emulate  everything that is different between the
> > > host and guest.
> > 
> > Directly assigning a device means you don't get to have differences between
> > the actual hardware device and what the guest sees.  The kind of thin
> > wrapper you're suggesting might have some use cases, but it's a different
> > problem from what we're trying to solve.
> 
> That's the problem. You've skipped several steps and gone startigh for 
> optimization before you've even got basic functionality working.

This is the basic functionality -- assign a piece of hardware to the
guest with minimal overhead.  Why go through contortions to construct some
intermediate phase that nobody's interested in using?

> You've also missed the point I was making.  In order to do device passthrough 
> you need to define a boundary allong which the emulated machine state can be 
> fully replicated on the host machine.  Anything inside this boundary is (by 
> definition) that same on both the host and guest systems (we're effectively 
> using host hardware to emulate a device for us). Outside that boundary the 
> host and guest systems will diverge.

I'm still not sure what the point is, then.  By directly assigning the
device the user is placing everything about the device on the "same as
host" side of that boundary.

We're not using host hardware to emulate a device, we're using host
hardware to send and receive packets under control of the guest.
Whatever hardware that is, the guest will deal with it, just as if the
guest weren't running in a vm.

> For a device that merely responds to CPU initiated MMIO transfers this is 
> pretty simple, it's the point at which MMIO transfers are generated. So the 
> guest gets a proxy device that intercepts accesses to that memory region, and 
> the host proxies some way for qemu to poke values at the host device.

The point is to be faster than virtio, not slower.  There would be no
reason for us to do this otherwise.

Emulating some specific device is not our goal, at all.  I realize that
that's a major part of what qemu does, but it's not the only thing it's
used for.

> > > Once you've done all the above, host device passthrough should be
> > > relatively straightforward.  Just replace the emulation bits in the
> > > above device with code that pokes at a real device via the relevant
> > > kernel API.
> > 
> > That's not what we mean by direct device assignment.
> 
> Maybe, but IMO but it's a necessary prerequisite. You're trying to run before 
> you can walk.

I disagree that it is a prerequisite.  It is a fundamentally different
thing, for a different purpose.

If it's a purpose that is important to you, and you think the proposed
config mechanisms don't accommodate that, then propose something that does.

> > We're talking about directly mapping the registers into the guest.  The
> > whole point is performance.
> 
> That's an additional step after you get passthrough working the normal way.

"normal"?

> We already have mechanisms (or at least patches) for mapping file-like 
> objects 
> into guest physical memory.  That's largely independent of device 
> passthrough.  
> It's a relatively minor tweak to how the passthrough device sets up its MMIO 
> regions.
> 
> Mapping host device MMIO regions into guest space is entirely uninteresting 
> unless we already have some way of creating guest-host passthrough devices.  

Isn't that what's being discussed?

> Creating guest-device passthrough devices isn't going to happen until the can 
> create arbitrary devices (within the set emulated by qemu) that interact with 
> the rest of the emulated machine in a similar way.

What do you mean by "interact with the rest of the emulated machine in a
similar way"?

-Scott