Re: Interface for SCSI transactions ?

[Thomas Schmitt]

Hi,

it looks like we need two fundamental decisions now, or else the
contraints become too fuzzy for developing the desired feature:

1: Shall the new call be implemented on the kernel side as a
   new method of struct device_emulation_ops ?

2: Shall the parameters be bundled in serialized structs as far as
   technically feasible ?
   Alternative:
   Each primitive data field shall be transferred as a separate RPC
   parameter.

I have planned for (yes, yes) but could re-arrange my sketch for (no, yes)
and (no, no). Nevertheless, some of my next TODO points depend on this
arrangement. It makes no sense to start working on them before a decision
is made.

Consequences:

Yes on 1:
Would implement a general way to cause driver specific transactions
on the device drivers of the kernel.

No on 1:
Then it would be a specialized call only for CD/DVD/BD drives.

Yes on 2:
New struct members could be added in a binary compatible way.

No on 2:
Only feasible together with No on 1, i.e. a specialized call.
More straightforward implementation of transmission. Not necessarily
leaner, but surely more easy to write and verify.
The RPC definition will have 20 to 30 parameters.
(The serializers of the yes case will be of similar complexity.)


------------------------------------------------------------------
Some detail arguments. Mine are not necessarily better than Olaf's.
I just play advocate for what i sketched so far.
------------------------------------------------------------------

Olaf Buddenhagen wrote:
> Yes, it's not possible to pass a complete structure as a sigle
> parameter, except by treating it as a flat byte (or int) array...
> [...]
> The *right* way of doing it is to list the elements of the structure as
> individual RPC parameters.

But this will cause a giant prototype. Very ugly to its users.

My current sketch has 9 members in the input struct and 18 in the
output one.
The leaner it is made, the more risk arises that it will later
need to be enlarged.
(I am much in favor of a stable ABI.)


> > Who ever has the plan to make a specialized driver in
> > userspace, will at least find an existing RPC gateway.
> No no no no! See my reply to Samuel --

I assume it's this one which reached me today:

Samuel Thibault wrote:
> > Why would it have to change again?

> Because RPC-over-RPC is inherently ugly,

> and I'm very sure that we don't
> want that in actual Hurd interfaces. I may (unwillingly) agree to have
> it like that in Mach, for consistency with the preexisting ugly
> device_get_status()/device_set_status() RPC-over-RPC Mach interfaces;
> but I will violently protest any such uglyness being introduced in
> actual Hurd interfaces.

I do not understand yet the "RPC-over-RPC" aspect, resp. why it is so bad.
To my understanding these calls transport generally defined parameters
to particular drivers which apply their particular interpretation
to these parameters.
This corresponds to quite common patterns of object orientation:
polymorphism and inheritance. Are these disliked ?

The generic nature of my current plan is a direct consequence of
Samuel's proposal/prescription to hook the call into the instances
of struct device_emulation_ops.
My current understanding is that a call in gnumach/linux/dev/glue/block.c
would also work without having a slot in struct device_emulation_ops.
It would just not be potentially applicable to all device classes
which have a device_emulation_ops interface.


> > Astounding that Linux SG_IO totally hides the auto sense mechanism,

> Hm... I don't know anything about this stuff; but my intuition is that
> if this *can* be hidden from users (without adverse effects), it
> should... Is it actually useful to have this data?

The user in this case is a device driver in userspace.
Any information about a failed transaction might be of interest.

The autosense feature on a failed command is a second SCSI/ATAPI
transaction which can fail itself for several reasons.
So i deem it worth to know whether it failed or succeeded.

But if the decision goes to an RFC prototype with non-aggregated
parameters, then it will hardly be desirable to have 6 more variables
in it.


> > Do you mean my plan to have serializer and de-serializer ?
> Sorry, I meant to mention the other term, but somehow forgot :-(

You did mention it. But it lasted half an hour of writing until
i came to its second usage. Until then i had forgotten your explanation.


> I'm very sceptical of any interface that doesn't use MIG's
> abilities here, instead doing the marshalling by hand.

/rant mode on/
I'd rather call it MIG's disabilities here.
It has neither means for aggregation, nor for a neutral representation
of data during transmission.
Representing arbitrary C-pointer graphs is quite impossible for an RPC
system. But MIG could at least provide what SUN RPC resp. XDR does.
/rant mode off/


> > So i think it is appropriate to prescribe an explizit representation
> > layer for the structs and their components.

> ...such as MIG .defs? ;-)

I would wholeheartedly agree, if MIG wasn't so dull.
A representation of struct components in the definition language would
allow conversions, if they become necessary in a distributed system.


> > If you really consider field sizes a problem, that's indeed one more
> >  reason for passing each field as an explicit RPC parameter (using a
> > machine-independent type).
> > 
> > I wouldn't worry about it too much though. While Mach was originally
> > designed to allow network-transparent IPC between machines using
> > different architectures,

The assumption of identical data representation on both sides of an RPC
lets me rise the question why to use RPCs at all and not shared memory
or even a shared address space. 

To my mind come at least two contemporary forms of mixed architectures:
The Cloud, and using graphics boards as number crunchers.


> The Linux sg_io structure seems to have a number of pointers to
> variable-sized arrays 

Only one of them, .dxferp, is really of large size and would best be
represented outside the struct. (In my sketch as two parameters
"in_data" and "out_data". In the CD use case they are mutally exclusive.)

The others, which i deem useful, can be represented as small byte
arrays of fixed length, or mapped into the struct serialization
as small byte arrays of variable size with few cost of copying.

------------------------------------------------------------------


Have a nice day :)

Thomas