Re: better buffer size for copy

[Robert Latham]

On Mon, Nov 21, 2005 at 12:45:40AM -0500, Phillip Susi wrote:
> If we are talking about the conventional blocking cached read,
> followed by a blocking cached write, then I think you will find that
> using a buffer size of several pages ( say 32 or 64 KB ) will be
> MUCH more efficient than 1024 bytes ( the typical local filesystem
> block size ), so using st_blksize for the size of the read/write
> buffer is not good.  I think you may be ascribing meaning to
> st_blksize that is not there. 

I mean no offense cutting out most of your points.  You describe great
ways to achieve high I/O rates for anyone writing a custom file mover.
I shouldn't have mentioned network file systems.  It's a distraction
from the real point of my patch: cp(1) should consider both the source
and the destination st_blksize.

All I expect from st_blksize is what the stat(2)
manpage suggests:

       The value st_blocks gives the size of  the  file  in  512-byte
       blocks.  (This  may  be  smaller than st_size/512 e.g. when the
       file has holes.) The value st_blksize gives the "preferred"
       blocksize for efficient file system  I/O.  (Writing to a file
       in smaller chunks may cause an inefficient
       read-modify-rewrite.)

All I really want is for cp(1) to do the right thing no matter what
the soruce or destination st_blksize value might be. 

In copying from a 4k blocksize file sytem to a 64k blocksize
filesystem, cp(1) will perform well, as it is using a 64k buffer.  

In copying *from* that 64k blocksize filesystem *to* a 4k blocksize
filesytem, cp(1) will not perform as well: it's using a 4k buffer and
so reading from the source filesystem in less-than-ideal chunks.

Thanks again for taking the time to respond.  I hope I have made the
intent of my patch more clear. 

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B