coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] cat, cp, mv, install, split: Set the minimum IO block size u


From: Bernhard Voelker
Subject: Re: [PATCH] cat, cp, mv, install, split: Set the minimum IO block size used, to 64KiB
Date: Thu, 21 Jul 2011 16:25:04 +0200
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110616 SUSE/3.1.11 Thunderbird/3.1.11

On 07/21/2011 03:59 PM, Pádraig Brady wrote:
On 21/07/11 13:15, Bernhard Voelker wrote:
On 07/21/2011 02:06 PM, Bernhard Voelker wrote:
On 07/21/2011 01:21 PM, Pádraig Brady wrote:
I'm a little wary of this being too aggressive,
but it does give a noticeable (13%) boost on my new laptop.
Here are the numbers from dd bs=$blksize if=/dev/zero of=/dev/null

blksize system-1 system-2
----------------------------
1024 734 MB/s 1.7 GB/s
2048 1.3 GB/s 3.0 GB/s
4096 2.4 GB/s 5.1 GB/s
8192 3.5 GB/s 7.3 GB/s
16384 3.9 GB/s 9.4 GB/s
32768 5.2 GB/s 9.9 GB/s
65536 5.3 GB/s 11.2 GB/s
131072 5.5 GB/s 11.8 GB/s
262144 5.7 GB/s 11.6 GB/s
524288 5.7 GB/s 11.4 GB/s
1048576 5.8 GB/s 11.4 GB/s


Hmm, I've taken the script from the test, but I'm not
quite sure about the numbers on my system:

$ for i in $(seq 0 10); do bs=$((1024*2**$i)); printf "%7s=" $bs;
src/timeout --foreground -sINT 1 dd bs=$bs if=/dev/zero of=/dev/null
2>&1 | sed -n 's/.* \([0-9.]* [GM]B\/s\)/\1/p' ; done
1024=2.3 GB/s
2048=4.0 GB/s
4096=6.7 GB/s
8192=9.7 GB/s
16384=16.3 GB/s
32768=31.3 GB/s
65536=52.2 GB/s
131072=81.6 GB/s
262144=115 GB/s
524288=141 GB/s
1048576=157 GB/s

I have a "Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz" system.
And I used `src/timeout` from the latest `git pull`.

Wow, what?
Total guess. Perhaps your hardware has facilities to allow the kernel
determine the buffers haven't been modified, thus avoiding
the data copy in the write to /dev/null ??
What kernel are you using?

genuine OpenSuSE 11.4: kernel 2.6.37.6-0.5-desktop,

Mine is 2.6.38.8-34.fc15.x86_64.
I wonder do the iflag=nocache oflag=nocache params change things?

nope:

for i in $(seq 0 15); do bs=$((1024*2**$i)); printf "%8s=" $bs; src/timeout --foreground -sINT 1 src/dd bs=$bs iflag=nocache oflag=nocache if=/dev/zero of=/dev/null 2>&1 | sed -n 's/.* \([0-9.]* [GM]B\/s\)/\1/p' ; done
    1024=1.8 GB/s
    2048=2.8 GB/s
    4096=4.2 GB/s
    8192=6.9 GB/s
   16384=12.0 GB/s
   32768=22.5 GB/s
   65536=40.0 GB/s
  131072=65.8 GB/s
  262144=96.4 GB/s
  524288=128 GB/s
 1048576=150 GB/s
 2097152=162 GB/s
 4194304=171 GB/s
 8388608=175 GB/s
16777216=175 GB/s
33554432=176 GB/s


The situation changes when I specify ibs and obs for dd
(I looped until 15) for testing):

address@hidden:~/git/coreutils>  for i in $(seq 0 15); do
bs=$((1024*2**$i)); printf "%8s=" $bs; src/timeout --foreground -sINT 1
src/dd ibs=$bs obs=$bs if=/dev/zero of=/dev/null 2>&1  | sed -n 's/.*
\([0-9.]* [GM]B\/s\)/\1/p' ; done
     1024=1.9 GB/s
     2048=3.0 GB/s
     4096=4.5 GB/s
     8192=5.9 GB/s
    16384=7.2 GB/s
    32768=7.8 GB/s
    65536=8.7 GB/s
   131072=9.3 GB/s
   262144=9.6 GB/s
   524288=9.8 GB/s
  1048576=9.9 GB/s
  2097152=6.0 GB/s
  4194304=5.1 GB/s
  8388608=4.6 GB/s
16777216=4.4 GB/s
33554432=4.3 GB/s

So you get more "standard" results with the data copy,
with a corresponding 11.5% increase when going from 32K ->  64K.
Interestingly, you only see a drop in performance at 1M ->  2M,
whereas I gradually drop performance starting at 64K ->  32K.

     1024=1.5 GB/s
     2048=2.7 GB/s
     4096=4.3 GB/s
     8192=5.9 GB/s
    16384=6.1 GB/s
    32768=5.9 GB/s
    65536=6.4 GB/s
   131072=6.1 GB/s
   262144=5.2 GB/s
   524288=4.2 GB/s
  1048576=4.3 GB/s
  2097152=4.2 GB/s
  4194304=3.4 GB/s
  8388608=3.3 GB/s
16777216=3.4 GB/s
33554432=3.4 GB/s

Do you think that the assumption - that the io block size of
a sample dd run can be transfered to cat, cp et al - is valid?

Have a nice day,
Berny



reply via email to

[Prev in Thread] Current Thread [Next in Thread]