Re: Another rfe: "cp" this time

[Pádraig Brady]

On 04/27/2012 06:15 PM, Bruce Korb wrote:
> Thank you for the suggestions.  I did finish my particular task with rsync,
> but I'd still like something that finishes in less than 10 hours.

Just to be clear, you started with cp, went home, resumed with rsync.
So rsync is not faster here, as there is no remote rsync server.

> It is longer
> than my work day.   Please note:  my only access is with a CIFS mount.
> *NO* shell access.  Thus, anything depending upon remote commands
> doesn't work.  This is not the only Windows-centric shop I've seen.

Sure, I was just comparing all options.

> 2012/4/27 Pádraig Brady <address@hidden>:
>>> I can either add this to cp, or roll my own toy.  What say you?
>>
>> Yes, cp is bad for high latency links as it has a
>> 32KiB buffer which it serially reads to and writes from.
> 
> The throughput is going to be dependent upon how much data CIFS
> sends without an ACK.  e.g. 1500 bytes would mean 20 round trips
> to fill the 30KB buffer.

If it is 1 large file you could optimize a bit like:

  dd bs=10MB if=/cifs/file of=/local/file

However as you say, if the cifs protocol is not
efficiently streaming over the high latency link
then there's not much we can do.

Note you could test your multiple readers idea simply like:

 (
   dd bs=32KiB skip=0000 seek=0000 count=1000 conv=notrunc &
   dd bs=32KiB skip=1000 seek=1000 count=1000 conv=notrunc &
   dd bs=32KiB skip=2000 seek=2000 count=1000 conv=notrunc
 ) > /local/file < /cifs/file

A simple wrapper could be used to start the correct number of dd processes.

cheers,
Pádraig.