Re: Another rfe: "cp" this time

[Pádraig Brady]

On 04/27/2012 06:47 PM, Pádraig Brady wrote:
> On 04/27/2012 06:15 PM, Bruce Korb wrote:
>> Thank you for the suggestions.  I did finish my particular task with rsync,
>> but I'd still like something that finishes in less than 10 hours.
> 
> Just to be clear, you started with cp, went home, resumed with rsync.
> So rsync is not faster here, as there is no remote rsync server.
> 
>> It is longer
>> than my work day.   Please note:  my only access is with a CIFS mount.
>> *NO* shell access.  Thus, anything depending upon remote commands
>> doesn't work.  This is not the only Windows-centric shop I've seen.
> 
> Sure, I was just comparing all options.
> 
>> 2012/4/27 Pádraig Brady <address@hidden>:
>>>> I can either add this to cp, or roll my own toy.  What say you?
>>>
>>> Yes, cp is bad for high latency links as it has a
>>> 32KiB buffer which it serially reads to and writes from.
>>
>> The throughput is going to be dependent upon how much data CIFS
>> sends without an ACK.  e.g. 1500 bytes would mean 20 round trips
>> to fill the 30KB buffer.
> 
> If it is 1 large file you could optimize a bit like:
> 
>   dd bs=10MB if=/cifs/file of=/local/file
> 
> However as you say, if the cifs protocol is not
> efficiently streaming over the high latency link
> then there's not much we can do.
> 
> Note you could test your multiple readers idea simply like:
> 
>  (
>    dd bs=32KiB skip=0000 seek=0000 count=1000 conv=notrunc &
>    dd bs=32KiB skip=1000 seek=1000 count=1000 conv=notrunc &
>    dd bs=32KiB skip=2000 seek=2000 count=1000 conv=notrunc
>  ) > /local/file < /cifs/file
> 
> A simple wrapper could be used to start the correct number of dd processes.
> 

Actually each dd process would have to maintain the file offsets
independently something like:

> file.out
(
 dd if=file.in of=file.out bs=32KiB skip=000 seek=000 count=100 conv=notrunc &
 dd if=file.in of=file.out bs=32KiB skip=100 seek=100 count=100 conv=notrunc &
 dd if=file.in of=file.out bs=32KiB skip=200 seek=200           conv=notrunc
 wait
)

Could you link to pcopy for reference?

cheers,
Pádraig.