[Bug ld/12682] New: [PATCH] high memory usage when linking many small object files.

[bertram.felgenhauer at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12682

           Summary: [PATCH] high memory usage when linking many small
                    object files.
           Product: binutils
           Version: 2.21
            Status: NEW
          Severity: normal
          Priority: P2
         Component: ld
        AssignedTo: address@hidden
        ReportedBy: address@hidden


Created attachment 5675
  --> http://sourceware.org/bugzilla/attachment.cgi?id=5675
patch against Debian's binutils-2.21.0.20110327

GNU ld takes a lot of memory when linking the Glasgow Haskell Compiler (ghc),
and many programs produced by this compiler. These programs have the
characteristic that they consist of many, literally thousands, small object
files with perhaps a dozen symbols each.

Version: GNU ld (GNU Binutils) 2.21.0.20110327

This topic comes up occasionally on haskell mailing lists, for example,
http://www.mail-archive.com/address@hidden/msg18215.html

I have a patch (attached below) that reduces the memory usage for linking ghc
from almost 440 MB to 190 MB on x86_64. I did not see any negative impact on
performance, but admittedly I did not try very hard to measure it (I really
expect no discernible impact: the extra work is limited to a couple of
allocations and copying of memory, and ld does quite a bit of other, heavier
lifting. The primary effect, namely the reduced working size of the program,
can only help performance.) For ghc, link time improved very slightly from
about 1.9 to 1.8 seconds.

The patch works by changing the default hashtable size of libbfd from 4k
entries to 31 entries, and increasing the granularity of hash table sizes by
adding more intermediate sizes. No code is changed at all. There is nothing
magical about the 31 except that going below 31 did not seem to improve memory
usage anymore.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.