Re: NMERGE hardcoded in sort.c

[Paul Courbis]

Unfortunately your download link gives :

Not Found
The requested URL /coreutils-4.5.3-alexa03.tar.gz was not found on this server.


--------------------------------------------------------------------------------

Apache/1.3.26 Server at alexautils.sourceforge.net Port 80

:-(

Regards

Paul

On Wed, Oct 01, 2003 at 12:59:10PM -0400, Andrew D Jewell wrote:
> You can get our version of coreutils 4.5.3 from 
> http://alexautils.sourceforge.net/
> 
> We've had good results with NMERGE up around 1000
> 
> The merge code in standard sort is O(NMERGE * total_lines) which is 
> already bad at 16 and terrible if you get much higher. The alexautils 
> code has different merging code which is O(log(NMERGE) * total_lines) 
> which works great.
> 
> adj
> 
> 
> At 12:57 PM +0200 9/30/03, Paul Courbis wrote:
> >Hi
> >
> >I'm using "sort" to, surprisingly, sort huge files (up to 2 GB). Of 
> >course I had to use the -S switch to tune memory usage. My concern 
> >is that I
> >don't have enough memory to sort in RAM, thus sort is using 
> >temporary files tat it merges to get the final result.
> >
> >Having a look to sort's sources I noticed that NMERGE is hardocoded 
> >(it's not even an option).
> >
> >My questions are :
> >
> >- I did a patch to make sort accept a new option (currently called 
> >-N to setup NMERGE). Is someone interested by this patch ?
> >- I'm doing some benches, but I already noticed that 16 (the 
> >hardcoded value) is not always the best one. For example my first 
> >serie of benches showed me that 18 is better in this particular 
> >case. Did someone worked on a model allowing to guess what would be 
> >the best value for NMERGE for a given set of data to sort ?
> >
> >Any hint would be apreciated
> >
> >regards
> >
> >Paul
> >
> >
> >
> >_______________________________________________
> >Bug-coreutils mailing list
> >address@hidden
> >http://mail.gnu.org/mailman/listinfo/bug-coreutils