RE: gsort problem (TAKE 2)

[Anthony Tiemens]

please ignore last email...

I inserted my backtrace after yours further down

Anthony

-----Original Message-----
From: Paul Eggert [mailto:address@hidden
Sent: Wednesday, 7 June 2006 3:33 PM
To: Anthony Tiemens
Cc: address@hidden
Subject: Re: gsort problem


"Anthony Tiemens" <address@hidden> writes:

> when I test it with a 10gig file, it fails with a "memory exhausted"
> error.  The unix admin has investigated and monitored the
> environment during the execution and appears there is plenty of
> memory etc.
>
> Any ideas what could cause this? Is it an incorrect compiler option
> or wrong compiler that may have been used?
>
> (The sorted output file gets to about 6.5gig before the failure)

That last point is odd.  Once 'sort' starts writing, it shouldn't
need to allocate much memory, since it's already done most of the
work.

How about if you run it under GDB, put a breakpoint on 'xalloc_die',
and then get a stack backtrace?

Your output might look something like the following, but with a
different stack trace (this stack trace is for a file that contains a
single line that is over 2 TB long, which I think 'sort' is entitled
to complain about :-):

=======

$ gdb src/sort
GNU gdb 6.4
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...Using host libthread_db 
library "/lib/libthread_db.so.1".

(gdb) b xalloc_die
Breakpoint 1 at 0x804eb56: file xalloc-die.c, line 37.
(gdb) r -o output biginput
Starting program: /home/eggert/src/gnu/cu/src/sort -o output biginput
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 28537)]
[Switching to Thread 16384 (LWP 28537)]

Breakpoint 1, xalloc_die () at xalloc-die.c:37
37        error (exit_failure, 0, "%s", _("memory exhausted"));
(gdb) where
#0  xalloc_die () at xalloc-die.c:37
#1  0x08050c08 in xnrealloc_inline (p=0x0, n=4231758848, s=1) at xmalloc.c:75
#2  0x08050c2d in xrealloc (p=0x401c0008, n=4231758848) at xmalloc.c:91
#3  0x08050cc1 in x2nrealloc_inline (p=0x401c0008, pn=0xbffff8ec, s=1)
    at xmalloc.c:177
#4  0x08050ce3 in x2realloc (p=0x401c0008, pn=0xbffff8ec) at xmalloc.c:195
#5  0x0804b47c in fillbuf (buf=0xbffff8e0, fp=0x8053ea8,
    file=0xbffffc49 "biginput") at sort.c:1075
#6  0x0804d6de in sort (files=0x8053e4c, nfiles=0,
    output_file=0xbffffc42 "output") at sort.c:1954
#7  0x0804eaf6 in main (argc=4, argv=0xbffffae4) at sort.c:2599
(gdb)

>>>>>>>>HERE'S MY BACKTRACE...
(gdb) ba
#0  0x125b8 in xalloc_die+0x18 ()
#1  0x19244 in xnrealloc_inline+0x7c ()
#2  0x19298 in xrealloc+0x30 ()
#3  0x193e0 in x2nrealloc_inline+0xd0 ()
#4  0x19430 in x2realloc+0x30 ()
#5  0xd4b4 in fillbuf+0x55c ()
#6  0xf560 in mergefps+0x748 ()
#7  0x1036c in merge+0xa4 ()
#8  0x10920 in sort+0x3b0 ()
#9  0x12224 in main+0x131c ()

This email and any attachments may contain privileged and confidential 
information and are intended for the named addressee only. If you have received 
this e-mail in error, please notify the sender and delete this e-mail 
immediately. Any confidentiality, privilege or copyright is not waived or lost 
because this e-mail has been sent to you in error. It is your responsibility to 
check this e-mail and any attachments for viruses.  No warranty is made that 
this material is free from computer virus or any other defect or error.  Any 
loss/damage incurred by using this material is not the sender's responsibility. 
 The sender's entire liability will be limited to resupplying the material.