bug-coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem b


From: Phillip Susi
Subject: Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB
Date: Tue, 03 Oct 2006 18:04:51 -0400
User-agent: Thunderbird 1.5.0.7 (Windows/20060909)

Tony Ernst wrote:
I believe the larger block sizes are especially beneficial with RAID. I'm adding Geoffrey Wehrman to the CC list, as he understands disk I/O much better than I do.

I believe most kernels always performs the actual IO in the same size chunks due to the block layer and cache, even if user space passes down a large buffer. The exception to this on Linux would be when you use O_DIRECT IO, then using buffer sizes at least as large as the stripe width is definitely good for keeping all the disks spinning.

In any case, the raid case falls under the pervue of the kernel buffer cache read-ahead mechanism, and is beyond the scope of coreutils. From the perspective of coreutils, using a larger buffer size has the benefit of reducing the number of system calls needed, but causes more memory to be locked down. As such you need to have some kind of limit on the size of the buffer so you don't try to exhaust system memory.

On the other hand, in a way it is up to the kernel to provide a reasonable value for the block size knowing full well that applications that use that value use it as a guide to the IO buffer size to use. In that case I suspect that XFS really should not be setting that value to many megabytes.


I don't think a cap is really necessary. This test and arbitrary limitation were put in to work around a specific problem on hpux. But it has side-effects that reach beyond hpux and the original problem. So the test should either be limited to hpux-only, or fixed to get rid of the side-effects by making it more specific to the original problem.







reply via email to

[Prev in Thread] Current Thread [Next in Thread]