Re: gsort problem

[Paul Eggert]

"Anthony Tiemens" <address@hidden> writes:

> when I test it with a 10gig file, it fails with a "memory exhausted"
> error.  The unix admin has investigated and monitored the
> environment during the execution and appears there is plenty of
> memory etc.
>
> Any ideas what could cause this? Is it an incorrect compiler option
> or wrong compiler that may have been used?
>
> (The sorted output file gets to about 6.5gig before the failure)

That last point is odd.  Once 'sort' starts writing, it shouldn't
need to allocate much memory, since it's already done most of the
work.

How about if you run it under GDB, put a breakpoint on 'xalloc_die',
and then get a stack backtrace?

Your output might look something like the following, but with a
different stack trace (this stack trace is for a file that contains a
single line that is over 2 TB long, which I think 'sort' is entitled
to complain about :-):

=======

$ gdb src/sort
GNU gdb 6.4
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...Using host libthread_db 
library "/lib/libthread_db.so.1".

(gdb) b xalloc_die
Breakpoint 1 at 0x804eb56: file xalloc-die.c, line 37.
(gdb) r -o output biginput
Starting program: /home/eggert/src/gnu/cu/src/sort -o output biginput
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 28537)]
[Switching to Thread 16384 (LWP 28537)]

Breakpoint 1, xalloc_die () at xalloc-die.c:37
37        error (exit_failure, 0, "%s", _("memory exhausted"));
(gdb) where
#0  xalloc_die () at xalloc-die.c:37
#1  0x08050c08 in xnrealloc_inline (p=0x0, n=4231758848, s=1) at xmalloc.c:75
#2  0x08050c2d in xrealloc (p=0x401c0008, n=4231758848) at xmalloc.c:91
#3  0x08050cc1 in x2nrealloc_inline (p=0x401c0008, pn=0xbffff8ec, s=1)
    at xmalloc.c:177
#4  0x08050ce3 in x2realloc (p=0x401c0008, pn=0xbffff8ec) at xmalloc.c:195
#5  0x0804b47c in fillbuf (buf=0xbffff8e0, fp=0x8053ea8, 
    file=0xbffffc49 "biginput") at sort.c:1075
#6  0x0804d6de in sort (files=0x8053e4c, nfiles=0, 
    output_file=0xbffffc42 "output") at sort.c:1954
#7  0x0804eaf6 in main (argc=4, argv=0xbffffae4) at sort.c:2599
(gdb)