Re: sort -m does too much work

[Jim Meyering]

Bo Borgerson <address@hidden> wrote:

> David Muir Sharnoff wrote:
>> I've got 200 1GB pre-sorted files.   If I try to merge
>> them with sort -m, it is obvioulsy trying to do too much
>> work: after running for a couple minutes, it has not
>> produced any output but it has made a 5 GB temporary
>> file.
>>
>> When the input is pre-sorted, no temporary file should
>> be required.
>>
>> Output should begin immediately.
>
>
> Hi David,
>
> The reason you're not seeing output immediately is because sort
> internally limits the number of files it will read at once.  By default
> this limit is set to 16.  When more files are to be merged, sort uses
> temporary files.
>
> Starting in release 7.0 this limit will be modifiable on the
> command-line using the --bath-size=N option.  With 200 files you'll

--batch-size=N, of course ;-)

If you'd like to try a test release which includes the new
feature, this is the latest:

  http://meyering.net/cu/coreutils-ss.tar.gz        8.8 MB
  http://meyering.net/cu/coreutils-ss.tar.lzma      3.7 MB