bug in bfd in open file cache

[Mikulas Patocka]

Hi

I have written linker, that uses bfd library to read object files and
produces an executable. Bfd keeps only few file descriptors open in a
cache and reopens files if they are not cached. However there is a bug in
this mechanism, when processing archives.

My linker actually opens all *.o files and *.a archives and walks through
them. When .a archive is closed because of file descriptor limit was
reched and an attempt to read from some of its object files is made, bfd
tries to open nonexistent file and crashes.

This is strace of a link process:

10921 brk(0x8327000)                    = 0x8327000
10921 read(5, "\0xmalloc\0xrealloc\0tparam1\0tg"..., 512) = 512
10921 lseek(5, 39424, SEEK_SET)         = 39424
10921 read(5, "\1\2\0\0\310\0\0\0\1\2\0\0\314\0"..., 512) = 324
10921 read(5, "", 512)                  = 0
10921 close(6)                          = 0
10921 munmap(0x4000d000, 4096)          = 0
10921 open("´­^ě^K", O_RDONLY)          = -1 ENOENT (No such file or
directory)
~?
~/.~?
10921 --- SIGSEGV (Segmentation fault) ---

backtrace in debugger shows this:

(gdb) bt
#0  0x4004bd48 in _IO_seekoff ()
#1  0x4004c45e in fseek ()
#2  0x8051881 in bfd_seek (abfd=0x8190a78, position=7820, direction=0) at
bfdio.c:328
^^^^^^ here bfd_seek calls fseek with NULL FILE * pointer --- and it
crashes
bfd_cached_lookup few lines above returned NULL because it coudn't open
file because the file was inside an archive.

#3  0x806a8f3 in bfd_elf_get_elf_syms (ibfd=0x8190a78,
symtab_hdr=0x81c4578, symcount=11,
    symoffset=0, intsym_buf=0x0, extsym_buf=0x831cf18, extshndx_buf=0x0)
at elf.c:433
#4  0x805e8c9 in bfd_elf32_slurp_symbol_table (abfd=0x8190a78,
symptrs=0x831cee8, dynamic=0)
    at elfcode.h:1103
#5  0x8071ea2 in _bfd_elf_get_symtab (abfd=0x8190a78, alocation=0x831cee8)
at elf.c:5634
#6  0x804b1b8 in check_if_usable (afilename=0x81908f8 "./lib/sh/LIBsh.A",
a=0x81909b0,
    b=0x8190a78) at TOOLS/LINK.C:803
^^^^^ here my linker tries to call bfd_canonicalize_symtab on a bfd, that
is a member of previously opened archive

#7  0x804b3f5 in search_archives () at TOOLS/LINK.C:842
#8  0x804cca3 in main (argc=64, argv=0xbffff404) at TOOLS/LINK.C:1538
#9  0x4002830a in __libc_start_main ()
(gdb)

The failing abfd looks like this:
(gdb) print *abfd
$3 = {id = 45, filename = 0x81bd378 "clktck.o", xvec = 0x809f650, iostream
= 0x0,
  cacheable = 0, target_defaulted = 0, lru_prev = 0x0, lru_next = 0x0,
where = 7338,
  opened_once = 0, mtime_set = 0, mtime = 0, ifd = 0, format = bfd_object,
  direction = read_direction, flags = 17, origin = 1654, output_has_begun
= 0,
  section_htab = {table = 0x81c6a70, size = 4051,
    newfunc = 0x8054780 <bfd_section_hash_newfunc>, memory = 0x81908e8},
  sections = 0x81c5a94, section_tail = 0x81c5d84, section_count = 6,
start_address = 0,
  symcount = 0, outsymbols = 0x0, dynsymcount = 0, arch_info = 0x80a3e04,
  arelt_data = 0x81bd330, my_archive = 0x81909b0, next = 0x0, archive_head
= 0x0,
  has_armap = 0, link_next = 0x0, archive_pass = 0, tdata = {aout_data =
0x81c4514,
    aout_ar_data = 0x81c4514, oasys_obj_data = 0x81c4514, oasys_ar_data =
0x81c4514,
    coff_obj_data = 0x81c4514, pe_obj_data = 0x81c4514, xcoff_obj_data =
0x81c4514,
    ecoff_obj_data = 0x81c4514, ieee_data = 0x81c4514, ieee_ar_data =
0x81c4514,
    srec_data = 0x81c4514, ihex_data = 0x81c4514, tekhex_data = 0x81c4514,
    elf_obj_data = 0x81c4514, nlm_obj_data = 0x81c4514, bout_data =
0x81c4514,
    mmo_data = 0x81c4514, sun_core_data = 0x81c4514, sco5_core_data =
0x81c4514,
    trad_core_data = 0x81c4514, som_data = 0x81c4514, hpux_core_data =
0x81c4514,
    hppabsd_core_data = 0x81c4514, sgi_core_data = 0x81c4514,
lynx_core_data = 0x81c4514,
    osf_core_data = 0x81c4514, cisco_core_data = 0x81c4514, versados_data
= 0x81c4514,
    netbsd_core_data = 0x81c4514, mach_o_data = 0x81c4514, mach_o_fat_data
= 0x81c4514,
    pef_data = 0x81c4514, pef_xlib_data = 0x81c4514, sym_data = 0x81c4514,
any = 0x81c4514},
  usrdata = 0x0, memory = 0x8190b20}

Note that this bfd is part of another bfd archive and cache-open
mechanisms do not expcet that.

Maybe GNU ld never keeps archives open too long and it walks them only in
one pass --- that's why the bug doesn't manifest with it.

I "fixed" the bug by increasing BFD_CACHE_MAX_OPEN to maximum value, but
it's not the correct solution.

Mikulas