Re: [Qemu-devel] virtio scsi host draft specification, v2

[Michael S. Tsirkin]

On Fri, May 20, 2011 at 10:21:03AM +0200, Paolo Bonzini wrote:
> Hi all,
> 
> here is the second version of the spec.  In the end I took the
> advice of merging all requestq's into one.  The reason for this is
> that I took a look at the vSCSI device and liked its approach of
> using SAM 8-byte LUNs directly.  While it _is_ complex (and not yet
> done right by QEMU---will send a patch for that), the scheme is
> actually quite natural to implement and use, and supporting generic
> bus/target/LUN topologies is good to have for passthrough, as well.
> 
> I also added a few more features from SAM to avoid redefining the
> structs in the future.
> 
> Of course it may be that I'm completely wrong. :)  Please comment on
> the spec!
> 
> Paolo

> Virtio SCSI Host Device Spec
> ============================
> 
> The virtio SCSI host device groups together one or more simple virtual
> devices (ie. disk), and allows communicating to these devices using the
> SCSI protocol.  An instance of the device represents a SCSI host with
> possibly many buses, targets and LUN attached.
> 
> The virtio SCSI device services two kinds of requests:
> 
> - command requests for a logical unit;
> 
> - task management functions related to a logical unit, target or
> command.
> 
> The device is also able to send out notifications about added
> and removed logical units.
> 
> v4:
>     First public version
> 
> v5:
>     Merged all virtqueues into one, removed separate TARGET fields

Document still seems to refer to multiple VQs - maybe I misunderstand?

> Configuration
> -------------
> 
> Subsystem Device ID
>     TBD
> 
> Virtqueues
>     0:control transmitq
>     1:control receiveq
>     2:requestq
> 
> Feature bits
>     VIRTIO_SCSI_F_INOUT - Whether a single request can include both
>         read-only and write-only data buffers.
> 
> Device configuration layout
>     struct virtio_scsi_config {
>     }
> 
>     (Still empty)
> 
> Device initialization
> ---------------------
> 
> The initialization routine should first of all discover the device's
> control virtqueues.
> 
> The driver should then place at least a buffer in the control receiveq.

Size of the buffer?


> Buffers returned by the device on the control receiveq may be referred
> to as "events" in the rest of the document.
> 
> The driver can immediately issue requests (for example, INQUIRY or
> REPORT LUNS) or task management functions (for example, I_T RESET).
> 
> Device operation: request queue
> -------------------------------
> 
> The driver queues requests to the virtqueue, and they are used by the device
> (not necessarily in order).
> 
> Requests have the following format:
> 
>     struct virtio_scsi_req_cmd {
>         u8 lun[8];
>         u64 id;
>         u8 task_attr;
>         u8 prio;
>         u8 crn;
>         u32 num_dataout, num_datain;
>         char cdb[];
>         char data[][num_dataout+num_datain];
>         u8 sense[];
>         u32 sense_len;
>         u32 residual;
>         u16 status_qualifier;
>         u8 status;
>         u8 response;
>     };
> 
>     /* command-specific response values */
>     #define VIRTIO_SCSI_S_OK              0
>     #define VIRTIO_SCSI_S_UNDERRUN        1
>     #define VIRTIO_SCSI_S_ABORTED         2
>     #define VIRTIO_SCSI_S_FAILURE         3
> 
>     The lun field addresses a bus, target and logical unit in the SCSI
>     host.  The id field is the command identifier as defined in SAM.
> 
>     The task_attr, prio field should always be zero, as task
>     attributes other than SIMPLE, as well as command priority, are
>     explicitly not supported by this version of the device.
>     CRN is also as defined in SAM; while it is generally expected to
>     be 0, clients can provide it.  The maximum CRN value defined by
>     the protocol is 255, since CRN is stored in an 8-bit integer.
> 
>     All of these fields are always read-only.
> 
>     The cdb, data and sense fields must reside in separate buffers.
>     The cdb field is always read-only.  The data buffers may be either
>     read-only or write-only, depending on the request, with the read-only
>     buffers coming first.  The sense buffer is always write-only.
> 
>     The request shall have num_dataout read-only data buffers and
>     num_datain write-only data buffers.  One of these two values must be
>     zero if the VIRTIO_SCSI_F_INOUT has not been negotiated.

Why do num_datain/num_dataout need to be there?
We can just look at the number of io/out bufs in
virtio descriptors, no?

Also, from experience, it's better not to have any layout
assumptions - let the guest stick everything in a single
in + single out buffer if it desires.


>     Remaining fields are filled in by the device.  The sense_len field
>     indicates the number of bytes actually written to the sense buffer,
>     while the residual field indicates the residual size, calculated as
>     data_length - number_of_transferred_bytes.

Again virtio gives you total number of written bytes in the used len
field.  So just one of these fields will be enough.

>     The status byte is written by the device to be the SCSI status code.
> 
>     The response byte is written by the device to be one of the following:
> 
>     - VIRTIO_SCSI_S_OK when the request was completed and the status byte
>       is filled with a SCSI status code (not necessarily "GOOD").
> 
>     - VIRTIO_SCSI_S_UNDERRUN if the content of the CDB requires transferring
>       more data than is available in the data buffers.
> 
>     - VIRTIO_SCSI_S_ABORTED if the request was cancelled due to a reset
>       or another task management function.
> 
>     - VIRTIO_SCSI_S_FAILURE for other host or guest error.
> 
> Device operation: control transmitq
> -----------------------------------
> 
> The control transmitq is used for other SCSI transport operations.
> These are not placed on the requestq so that they can be sent out-of-band,
> even when the requestq is full.  This is particularly important for task
> management functions.
> 
> Requests have the following format:
> 
>     struct virtio_scsi_ctrl
>     {
>         u32 type;
>         ...
>         u8 response;
>     }
> 
>     The type identifies the remaining fields.
> 
> The following commands are defined:
> 
> - Task management function
> 
>     #define VIRTIO_SCSI_T_TMF                      0
> 
>     #define VIRTIO_SCSI_T_TMF_ABORT_TASK           0
>     #define VIRTIO_SCSI_T_TMF_ABORT_TASK_SET       1
>     #define VIRTIO_SCSI_T_TMF_CLEAR_ACA            2
>     #define VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET       3
>     #define VIRTIO_SCSI_T_TMF_I_T_NEXUS_RESET      4
>     #define VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET   5
>     #define VIRTIO_SCSI_T_TMF_QUERY_TASK           6
>     #define VIRTIO_SCSI_T_TMF_QUERY_TASK_SET       7
> 
>     struct virtio_scsi_ctrl_tmf
>     {
>         u32 type;
>         u32 subtype;
>         u8 lun[8];
>         u64 id;
>         u8 additional[];
>         u8 response;
>     }
> 
>     /* command-specific response values */
>     #define VIRTIO_SCSI_S_FUNCTION_COMPLETE        0
>     #define VIRTIO_SCSI_S_FAILURE                  3
>     #define VIRTIO_SCSI_S_FUNCTION_SUCCEEDED       4
>     #define VIRTIO_SCSI_S_FUNCTION_REJECTED        5
>     #define VIRTIO_SCSI_S_INCORRECT_LUN            6
> 
>     The type is VIRTIO_SCSI_T_TMF.  All fields but the last one are
>     filled by the driver, the response field is filled in by the device.
>     The id command must match the id in a SCSI command.  Irrelevant fields
>     for the requested TMF are ignored.
> 
>     Note that since ACA is not supported by this version of the spec,
>     VIRTIO_SCSI_T_TMF_CLEAR_ACA is always a no-operation.
> 
>     The outcome of the task management function is written by the device
>     in the response field.  Return values map 1-to-1 with those defined
>     in SAM.
> 
> - Asynchronous notification query
> 
>     #define VIRTIO_SCSI_T_AN_QUERY                    1
> 
>     struct virtio_scsi_ctrl_an {
>         u32 type;
>         u8  lun[8];
>         u32 event_requested;
>         u32 event_actual;
>         u8  response;
>     }
> 
>     #define VIRTIO_SCSI_EVT_ASYNC_MEDIA_CHANGE        16
> 
>     By sending this command, the driver asks the device which events
>     the given LUN can report, as described in Annex A of the SCSI MMC
>     specification.  The driver writes the events it is interested in
>     into the event_requested; the device responds by writing the events
>     that it supports into event_actual.
> 
>     The type is VIRTIO_SCSI_T_AN_QUERY.  The lun and event_requested
>     fields are written by the driver.  The event_actual and response
>     fields are written by the device.
> 
>     Valid values of the response byte are VIRTIO_SCSI_S_OK or
>     VIRTIO_SCSI_S_FAILURE (with the same meaning as above).
> 
> - Asynchronous notification subscription
> 
>     #define VIRTIO_SCSI_T_AN_SUBSCRIBE                2
> 
>     struct virtio_scsi_ctrl_an {
>         u32 type;
>         u8  lun[8];
>         u32 event_requested;
>         u32 event_actual;
>         u8  response;
>     }
> 
>     #define VIRTIO_SCSI_EVT_ASYNC_MEDIA_CHANGE        16
> 
>     By sending this command, the driver asks the specified LUN to report
>     events for its physical interface, as described in Annex A of the SCSI
>     MMC specification.  The driver writes the events it is interested in
>     into the event_requested; the device responds by writing the events
>     that it supports into event_actual.
> 
>     The type is VIRTIO_SCSI_T_AN_SUBSCRIBE.  The lun and event_requested
>     fields are written by the driver.  The event_actual and response
>     fields are written by the device.
> 
>     Valid values of the response byte are VIRTIO_SCSI_S_OK,
>     VIRTIO_SCSI_S_FAILURE (with the same meaning as above).
> 
> Device operation: control receiveq
> ----------------------------------
> 
> The control receiveq is used by the device to report information on
> logical units that are attached to it.  The driver should always
> leave a few (?) buffers ready in the control receiveq.  The device may
> end up dropping events if it finds no buffer ready.

Is this safe? It looks like there's a finite number of possible events.
Can't the device queue them until there's a buffer?
If this mechanism is unreliable, how is it useful?


> Buffers are placed in the control receiveq and filled by the device when
> interesting events occur.  Events have the following format:
> 
>     #define VIRTIO_SCSI_T_EVENTS_MISSED   0x80000000
> 
>     struct virtio_scsi_ctrl_recv {
>         u32 event;
>         ...
>     }
> 
> If bit 31 is set in the event, the device failed to report an event due
> to missing buffers.  In this case, the driver should poll the logical
> units for unit attention conditions, and/or do whatever form of bus scan
> is appropriate for the guest operating system.
> 
> The following events are defined:
> 
> - Transport reset
> 
>     #define VIRTIO_SCSI_T_TRANSPORT_RESET  0
> 
>     struct virtio_scsi_reset {
>         u32 event;
>         u8  lun[8];
>         u32 reason;
>     }
> 
>     #define VIRTIO_SCSI_EVT_RESET_HARD         0
>     #define VIRTIO_SCSI_EVT_RESET_RESCAN       1
>     #define VIRTIO_SCSI_EVT_RESET_SHUTDOWN     2
>     #define VIRTIO_SCSI_EVT_RESET_REMOVED      3
> 
>     By sending this event, the device signals that a logical unit
>     on a target has been reset, including the case of a new device
>     appearing or disappearing on the bus.
> 
>     The device fills in all fields.  The event field is set to
>     VIRTIO_SCSI_T_TRANSPORT_RESET.  The following format is used
>     to represent an event that affects all LUNs enumerated by
>     a target:
> 
>     - for logical units at the top level ("bus 0"), the LUN should be
>     (255,255,0,0,0,0,0,0), whose meaning is "logical unit not specified";
> 
>     - otherwise it should identify a bus and target indication in
>     peripheral device addressing format, followed by the six bytes
>     (255,255,0,0,0,0), whose meaning is also "logical unit not
>     specified".
> 
>     The reason value is one of the four #define values appearing above.
>     VIRTIO_SCSI_EVT_RESET_REMOVED is used if the target or logical unit
>     is no longer able to receive commands.  VIRTIO_SCSI_EVT_RESET_HARD
>     is used if the logical unit has been reset, but is still present.
>     VIRTIO_SCSI_EVT_RESET_RESCAN is used if a target or logical unit has
>     just appeared on the device.  VIRTIO_SCSI_EVT_RESET_SHUTDOWN
>     is used when the host wants to initiate a graceful shutdown of a
>     logical unit.
> 
>     Events should also be reported via sense codes or response codes
>     (this obviously does not apply to newly appeared buses or targets,
>     since the application has never discovered them):
> 
>     - VIRTIO_SCSI_EVT_RESET_HARD
>       sense UNIT ATTENTION
>       asc POWER ON, RESET OR BUS DEVICE RESET OCCURRED
> 
>     - VIRTIO_SCSI_EVT_RESET_RESCAN
>       sense UNIT ATTENTION
>       asc REPORTED LUNS DATA HAS CHANGED
> 
>     - VIRTIO_SCSI_EVT_RESET_SHUTDOWN
>       sense UNIT ATTENTION
>       asc TARGET OPERATING CONDITIONS HAVE CHANGED
>       ascq 0x80 (vendor specific)
> 
>     - VIRTIO_SCSI_EVT_RESET_REMOVED
>       sense ILLEGAL REQUEST
>       asc LOGICAL UNIT NOT SUPPORTED
> 
>     However, in general events should be more easily handled by the
>     driver than sense codes.
> 
> - Asynchronous notification
> 
>     #define VIRTIO_SCSI_T_ASYNC_NOTIFY     1
> 
>     struct virtio_scsi_an_event {
>         u8  lun[8];
>         u32 event;
>     }
> 
>     #define VIRTIO_SCSI_EVT_ASYNC_MEDIA_CHANGE        16
> 
>     By sending this event, the device signals that an event was
>     fired from a physical interface.  The device only sends events
>     that the driver has subscribed to via the "Asynchronous notification
>     subscription" command.
> 
>     All fields are written by the device.  The event field is set to
>     VIRTIO_SCSI_T_ASYNC_NOTIFY.

We'll have to define events, right?