Re: [Qemu-devel] [RFC PATCH v3 3/3] VFIO Type1 IOMMU change: to support with iommu and without iommu

[Dong Jia]

On Tue, 7 Jun 2016 02:47:10 +0000
"Tian, Kevin" <address@hidden> wrote:

> > From: Dong Jia
> > Sent: Monday, June 06, 2016 2:59 PM
> > 
> 
> [...]
> 
> > Channel I/O is quite different to PCI, so I copied some more details
> > here. Hope these could help.
> > 
> > Channel subsystem:
> > The channel subsystem directs the flow of information between I/O devices
> > and main storage. It relieves CPUs of the task of communicating directly
> > with I/O devices and permits data processing to proceed concurrently with
> > I/O processing.
> > 
> > Channel path:
> > The channel subsystem uses one or more channel paths as the communication
> > link in managing the flow of information to or from I/O devices.
> > 
> > Subchannel:
> > Within the channel subsystem are subchannels. One subchannel of type I/O
> > is provided for and dedicated to each I/O device accessible to the channel
> >                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > subsystem.
> > 
> > Control unit:
> > A control unit implements a standardized interface which translates between
> > the Channel Subsystem and the actual device. It does this by adapting the
> > characteristics of each device so that it can respond to the standard form
> > of control provided by the channel subsystem.
> > 
> > Channel Path:
> > The channel subsystem communicates with the I/O devices by means of
> > channel paths between the channel subsystem and control units.
> > 
> > +-------------------+
> > | channel subsystem |
> > +-------------------+
> > |                   |
> > |   +----------+    |              +--------------+    +------------+
> > |   |subchannel|    | channel path | control unit |    | I/O device |
> > |   +---------------------------------------------------------------+
> > |   | subchno  |    |              |              |    |    devno   |
> > |   +----------+    |              +--------------+    +------------+
> > |                   |
> > +-------------------+
> > 
> > There is no concept of ccw-device registers by the subchannel. Control
> > unit will interact with the device, collect the I/O result, and inform
> > the result to the subchannel.
> > So it seems to me, there is no needs to provide region info for
> > isolation. As mentioned above, the isolation is quite natural.
> > 
> > Please correct me in case I misunderstood some of the concepts in your
> > questions and gave irrelevant answers. :>
> > 
> 
> Thanks for above background which is very useful. Several follow-up Qs:
> 
> 1) Does it mean that VFIO is managing resource in subchannel level, so Qemu
> can only operate subchannels assigned to itself
Dear Kevin,

This understanding is basically right, but not that exactly.

Linux creates a 'struct ccw_device' for each device it has detected, and
a 'struct subchannel' for the corresponding subchannel. When we issue
a command to a device instance, the device can find the subchannel and
pass the command to it.

The current vfio-ccw implementation targets a device passthru. So I'd say
that VFIO is managing resource in both the device and the
subchannel level.

However, there is a discussion inside our team that tries to figure out
if a subchannel passthru would be better. So, in the future, it's
possible to do the management in a subchannel level only.

> (then emulate as the complete channel io sub-system to guest)?
This is right.

> 
> 2) How are ccw commands associated with a subchannel?
The I/O instruction requires a subchannel id and an ORB
(Operation-Request Block, which contains the execution parameters,
including the address of the ccws). So when an I/O instruction was
intercepted, we know which is the target subchannel. And using this
target information, we can find the real physical device and the
subchannel to perform the instruction.

> Are they submitted
> through a dedicated subchannel interface (so VFIO can easily map that 
> interface)
We can understand it this way.

> or that subchannel is specified by a special ccw cmd (means VFIO-ccw needs
> to scan cmds to avoid malicious attempts on non-assigned subchannels)?
No. CCWs themselves don't contain subchannel information.

> 
> Thanks
> Kevin
> 

--------
Dong Jia