Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

[Pádraig Brady]

On 02/03/10 06:32, Joey Degges wrote:
> 2010/3/1 Joey Degges<address@hidden>
>
> > Thanks for testing, these results are interesting. I will run some tests on
> > various systems tonight and report back. Maybe my earlier results were
> > caused by peculiar hardware.
> Results differ once again, this time I've tested on a different system.
>
> Here is pastebin of the code I used to spin. Its some math in a large loop,
> should run for ~5 minutes (enough time to do most tests):
> http://codepad.org/tCtjLw7H
>
> CPU: Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
> RAM: 2G
> kernel: 2.6.31-19-generic #56-Ubuntu SMP x86_64
> input file: 97M, gensort -a 1000000 file
> cmd: umount /mnt&&  mount /dev/foo /mnt&&  /usr/bin/time -v sort<  file>
> /dev/null
>
> mechanical disk @ 80MB/s, ext3 - 4x spinning processes
> mode      :   real, %CPU, voluntary/involuntary context switches
> NORMAL    : 14.73s,  10%, 408/34
> SEQUENTIAL: 10.95s,  13%, 220/52
> WILLNEED  : 05.22s,  35%, 026/57
>
> flash key @ 20MB/s, ext4 - 4x spinning processes
> NORMAL    : 88.49s,  01%, 1521/42
> SEQUENTIAL: 69.76s,  02%, 1131/57
> WILLNEED  : 50.29s,  03%, 0174/31
>
> flash key @ 11MB/s, FAT32 - 4x spinning processes
> NORMAL    : 106.5s,  01%, 1334/32
> SEQUENTIAL: 85.63s,  01%, 0765/53
> WILLNEED  : 68.70s,  02%, 0193/59
>
> mechanical disk @ 80MB/s, ext3
> NORMAL    : 02.70s,  64%, 748/9
> SEQUENTIAL: 02.71s,  61%, 374/8
> WILLNEED  : 02.79s,  57%, 026/5
>
> flash key @ 20MB/s, ext4
> NORMAL    : 06.25s,  25%, 1505/8
> SEQUENTIAL: 06.18s,  26%, 1136/11
> WILLNEED  : 06.10s,  26%, 0156/8
>
> flash key @ 11MB/s, FAT32
> NORMAL    : 10.23s,  15%, 1320/7
> SEQUENTIAL: 10.12s,  16%, 0762/7
> WILLNEED  : 10.14s,  16%, 0193/4
>
>
> Note the large difference in the number of voluntary context switches
> between the different modes. This number appears to only be dependent on the
> device being read from and the advice mode, not the system load. When the
> system is heavily loaded the process that context switches the least will be
> able to consume more resources. From these results you can see that
> SEQUENTIAL reduces the voluntary context switching from 25-46% and WILLNEED
> reduces the voluntary context switching from 85-96%. This also explains why
> you won't notice the improvements due to SEQUENTIAL or WILLNEED very much in
> unburdened systems.

Thanks for the thorough testing.
That confirms the context switch is done when the read()
needs data to be retrieved from disk, and that it has a
large affect on your sort on multicore, whereas it has
no affect on my single core.

I'm still unsure as to whether we should enable WILLNEED
as I've got a counter point in another email I'll send in a while.
Also I'm worried in the low RAM case that WILLNEED will thrash
other caches, perhaps useful to sort itself if it's processing
multiple files or running in merge mode. I'll do some more
testing with a view to enabling WILLNEED, but in any case
SEQUENTIAL should also be enabled so that readahead is done
on any portion of the file that it's not possible to
prepopulate the cache with.

cheers,
Pádraig.