Re: Using storage offloading capabilities

[Pádraig Brady]

On 03/24/2014 02:24 PM, Federico Simoncelli wrote:
> Hi everyone,
>  I am evaluating the possibility to add some storage offloading
> capabilities to existing utilities.
> For example I'd be interested in leveraging WRITE SAME and EXTENDED
> COPY scsi commands to initialize and copy data on iscsi/fcp devices.
> 
> It seems that both commands could be easily added to dd, WRITE SAME
> for example could be implemented as the "sparse" conv (detecting
> zeroes in the input stream) and EXTENDED COPY can be attempted when
> input and output devices are sg devices.
> 
> There's also a more ambitious goal of adding support for the device
> mapper (e.g. lvm logical volumes) reading the dm table and using the
> correct offsets to offload the initialization and the copy to the
> storage. I already did a quite extensive analysis on how to actually
> implement that but I don't want to mix too many subjects at once.
> 
> As initial step I'd like to understand if there's some interest in
> adding offloading support to dd.

So I see there is a more specific tool for this:
http://sg.danny.cz/sg/ddpt.html

Now I agree that it would be good for dd (and other utils)
to take advantage of this functionality where possible.
However ideally it would be a a higher more general level.

Take copy offloading for example. That would best exposed
_by the system_ through a copyfile() call, which could
do the appropriate thing depending on source and dest,
without pushing details of that into each application.
That system call has already been discussed in Linux at least.

For doing this at the block level, there might be
an analogous copyrange() call that could work similarly.

For efficiently reading zeros there is the lseek(...,SEEK_HOLE)
interface and for writing zeros fallocate(...) might suffice.
At the block level there could be analogous calls for this too.

The coretuils could attempt to use these generic interfaces if possible,
falling back to more traditional interfaces if not supported.

So in summary it seems a little too specific to attempt
iscsi specific stuff within dd.

thanks,
Pádraig.