Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

[Pádraig Brady]

On 01/03/10 09:36, Pádraig Brady wrote:
> On 28/02/10 00:46, Pádraig Brady wrote:
> > On 27/02/10 22:16, Joey Degges wrote:
> > > 2010/2/27 Pádraig Brady <address@hidden <mailto:address@hidden>>
> > >
> > > Well as long as the reads are big enough,
> > > read-process-read-process-... won't be significantly different than
> > > read-read-process-process-... However in the case where the
> > > storage and CPU are contended for by other processes the former
> > > is more desirable. You could test that by spinning the CPU with
> > > another process while doing the test. It still seems to me
> > > that WILLNEED is not appropriate for the sequential case.
> > >
> > > Here are the results of sorting when two other processes are running at
> > > 100% each (dual core) for the duration of the sort:
> > > NORMAL : 279.03s, 12.0% CPU
> > > SEQUENTIAL: 230.92s, 14.5% CPU
> > > WILLNEED : 186.26s, 18.5% CPU
> > >
> > > I suspect this trend may also repeat in situations where other processes
> > > are contending for both CPU and storage. Perhaps the priority of the
> > > WILLNEED read ahead is much higher than that of typical reads? Just a
> > > guess.
> >
> > Interesting results, thanks!
> > I'll play around with this with different buffer
> > sizes and concurrent loads to see how it behaves
> > on my single core system.
>
> Will check WILLNEED v SEQUENTIAL this evening.

TBH I couldn't see why you got such good results for WILLNEED,
especially since sort (with enough RAM) will read all the
data up front anyway. In my testing I did not get a benefit
from using WILLNEED over SEQUENTIAL. I still think SEQUENTIAL
is more appropriate and will allow sort to better share the
system resources.
  I have confirmed that WILLNEED reads the whole file synchronously,
which is surprising to me as I thought it should indicate to the
system to cache the file when possible rather than immediately.
I'll ask the kernel guys to see whether one also needs a "background"
tunable to get it running asynchronously. I'm also thinking that
the synchronous reading may not be desired in merge mode for example.
  Perhaps your WILLNEED was giving a better cpu affinity
characteristics than the other modes on your dual core system?

My system details and results are:

CPU:    pentium m single core 1.7GHz
RAM:    2G
kernel: 2.6.30.10-105.2.23.fc11.i586
cmd:    time LANG=C sort < file > /dev/null

88MB file of random numbers. mechanical disk @ 34 MB/s
 NORMAL      31.6 (26.7 user)
 SEQUENTIAL  31.6
 WILLNEED    31.6

88MB file of random numbers. flash key @ 21 MB/s
 NORMAL      31.6 (26.8 user)
 SEQUENTIAL  27.7
 WILLNEED    27.7

88MB file of random numbers. flash key @ 21 MB/s + spinning process
 NORMAL      59.5 (27.0 user)
 SEQUENTIAL  59.5
 WILLNEED    59.5

32MB file of random numbers on 800KB/s compact flash
 NORMAL      45.6 (7.8 user)
 SEQUENTIAL  45.2
 WILLNEED    45.2

32MB file of random numbers on 800KB/s compact flash + spinning process
 NORMAL      53.7 (7.8 user)
 SEQUENTIAL  53.7
 WILLNEED    54.9

So we can see that the only benefit under the conditions above
is the increased readahead on the fast flash device where the
wall clock time was reduced by 17% for you and 12% for my flash key.
So that's significant and worth enabling SEQUENTIAL mode for.

I'll push a version with just SEQUENTIAL enabled soon,
unless I see results to the contrary (with an explanation :))

cheers,
Pádraig.