I had posted the following as a guile bug by mistake. I am reposting it here with the hope that more people would be able to take a look at it and comment...
Hi,
We have observed the following error a few times with guile
2.0.11 (32-bit) on x86_64 Linux while processing large binary files
(1.5GB+) and comparing messages contained therein.
fish: hashtab.c:137: vacuum_weak_hash_table: Assertion `removed <= len' failed.
I
am appending the stack trace below. There is one instance where we can
consistently reproduce this problem. We see this issue only when we
have threading enabled (using future construct) to compare the messages
in parallel. Reading of data from the binary files is on a single
thread only. We don't get this error when we run everything on a single
thread, even though the memory footprint does grow.
I found references to a similar problem in the links below –
http://wingolog.org/archives/2011/02/25/ports-weaks-gc-and-dark-matterhttps://www.gnunet.org/bot/log/guile/2014-02-04In
this case also we see consistent memory footprint increase, even though
we probably are not opening/closing many ports and should not be
leaking memory. We also confirmed that in this case we are not running
out of virtual address space.
Given that we can consistently
reproduce this problem on a particular system, is there something we could try to find the underlying cause of this problem? Does anyone have an explanation for why using future construct appears to trigger
this problem (at least sooner)?
Thanks,
Anand
#0 0x00c8c430 in __kernel_vsyscall ()
#1 0x003bfb01 in raise () from /lib/libc.so.6
#2 0x003c13da in abort () from /lib/libc.so.6
#3 0x003b8ddb in __assert_fail_base () from /lib/libc.so.6
#4 0x003b8e96 in __assert_fail () from /lib/libc.so.6
#5 0x080e93d7 in vacuum_weak_hash_table (table=0x97f8b10) at hashtab.c:137
#6 0x080e9857 in weak_gc_callback (hook_data=0x0, fn_data=0x6, data="" at hashtab.c:437
#7 weak_gc_hook (hook_data=0x0, fn_data=0x6, data="" at hashtab.c:446
#8 0x080eaa2d in scm_c_hook_run (hook=0x895acc0, data="" at hooks.c:103
#9 0x080dfda6 in run_before_gc_c_hook () at gc.c:240
#10 0x08187719 in GC_notify_full_gc (stop_func=0x81868e0 <GC_never_stop_func>) at alloc.c:334
#11 GC_try_to_collect_inner (stop_func=0x81868e0 <GC_never_stop_func>) at alloc.c:429
#12 0x08187d1b in GC_collect_or_expand (needed_blocks=4, ignore_off_page=0, retry=0) at alloc.c:1242
#13 0x0818a64f in GC_alloc_large (lb=14080, k=1, flags=0) at malloc.c:63
#14 0x0818a9c2 in GC_generic_malloc (lb=14076, k=1) at malloc.c:175
#15 0x0818ace7 in GC_core_malloc (lb=14076) at malloc.c:263
#16 0x080df572 in do_gc_malloc (size=0, what=0x6 <Address 0x6 out of bounds>) at gc-malloc.c:100
#17 0x0813a565 in scm_c_make_vector (k=3517, fill=0x304) at vectors.c:408
#18
0x080e9471 in scm_i_rehash (table=0x97f8b10, hash_fn=0x816c1d0
<scm_ihashq>, closure=0x0, func_name=0x854f8bd
"scm_hash_fn_create_handle_x") at hashtab.c:344
#19 0x080e9e23 in
scm_hash_fn_create_handle_x (table=0x97f8b10, obj=0xc41c638, init=0x4,
hash_fn=0x816c1d0 <scm_ihashq>, assoc_fn=0x80c7eb0
<scm_sloppy_assq>, closure=0x0)
at hashtab.c:748
#20
0x080ea11d in scm_hash_fn_set_x (table=0x97f8b10, obj=0xc41c638,
val=0x4, hash_fn=0x816c1d0 <scm_ihashq>, assoc_fn=0x80c7eb0
<scm_sloppy_assq>, closure=0x0)
at hashtab.c:804
#21 0x080ea2a7 in scm_hashq_set_x (table=0x97f8b10, key=0xc41c638, val=0x4) at hashtab.c:948
#22 0x08108aee in scm_new_port_table_entry (tag=381) at ports.c:688
#23 0x0813207e in scm_mkstrport (pos=0x2, str=0x4, modes=327680, caller=0x855b842 "call-with-output-string") at strports.c:288
#24 0x08132393 in scm_call_with_output_string (proc=0xbbf49e0) at strports.c:427
#25 0x08143d21 in vm_regular_engine (vm=0x9821060, program=0x0, argv=0x1, nargs=0) at vm-i-system.c:855
#26 0x0813c367 in scm_c_vm_run (vm=0x9821060, program=0xa267de0, argv=0x0, nargs=0) at vm.c:768
#27 0x0813c3cd in scm_load_compiled_with_vm (file=0xa267e60) at vm.c:1079
#28 0x08143d21 in vm_regular_engine (vm=0x9821060, program=0x0, argv=0x1, nargs=1) at vm-i-system.c:855
#29 0x0813c367 in scm_c_vm_run (vm=0x9821060, program=0x9868470, argv=0xffe5ec50, nargs=1) at vm.c:768
#30 0x080d9036 in scm_primitive_eval (exp=0xa261938) at eval.c:692
#31 0x080d908d in scm_eval (exp=0xa261938, module_or_state=0x98b4630) at eval.c:726
#32 0x08117ac2 in scm_shell (argc=12, argv=0xffe5ed94) at script.c:439
#33 0x08068c4f in main (argc=12, argv=0xffe5ed94) at fish.cpp:90