[debbugs-tracker] bug#19180: closed (vacuum_weak_hash_table error)

[GNU bug Tracking System]

Your message dated Tue, 12 Jul 2016 09:32:16 +0200
with message-id <address@hidden>
and subject line Re: bug#19180: guile bug#19180: vacuum_weak_hash_table error
has caused the debbugs.gnu.org bug report #19180,
regarding vacuum_weak_hash_table error
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden)


-- 
19180: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19180
GNU Bug Tracking System
Contact address@hidden with problems

--- Begin Message --- Subject: vacuum_weak_hash_table error Date: Tue, 25 Nov 2014 16:53:28 +0530

Hi,

We have observed the following error a few times with guile 2.0.11 (32-bit) on x86_64 Linux while processing large binary files (1.5GB+) and comparing messages contained therein. 

fish: hashtab.c:137: vacuum_weak_hash_table: Assertion `removed <= len' failed. 

I am appending the stack trace below.  There is one instance where we can consistently reproduce this problem.  We see this issue only when we have threading enabled (using future construct) to compare the messages in parallel.  Reading of data from the binary files is on a single thread only.  We don't get this error when we run everything on a single thread, even though the memory footprint does grow.

I found references to a similar problem in the links below –

http://wingolog.org/archives/2011/02/25/ports-weaks-gc-and-dark-matter
https://www.gnunet.org/bot/log/guile/2014-02-04

In this case also we see consistent memory footprint increase, even though we probably are not opening/closing many ports and should not be leaking memory.  We also confirmed that in this case we are not running out of virtual address space.

Given that we can consistently reproduce this problem on a particular system, is there something you would like us to try to find the underlying cause of this problem?  Do you have an explanation why using future construct appears to trigger this problem (at least sooner)?

Thanks,
Anand


#0  0x00c8c430 in __kernel_vsyscall ()
#1  0x003bfb01 in raise () from /lib/libc.so.6
#2  0x003c13da in abort () from /lib/libc.so.6
#3  0x003b8ddb in __assert_fail_base () from /lib/libc.so.6
#4  0x003b8e96 in __assert_fail () from /lib/libc.so.6
#5  0x080e93d7 in vacuum_weak_hash_table (table=0x97f8b10) at hashtab.c:137
#6  0x080e9857 in weak_gc_callback (hook_data=0x0, fn_data=0x6, data="" at hashtab.c:437
#7  weak_gc_hook (hook_data=0x0, fn_data=0x6, data="" at hashtab.c:446
#8  0x080eaa2d in scm_c_hook_run (hook=0x895acc0, data="" at hooks.c:103
#9  0x080dfda6 in run_before_gc_c_hook () at gc.c:240
#10 0x08187719 in GC_notify_full_gc (stop_func=0x81868e0 <GC_never_stop_func>) at alloc.c:334
#11 GC_try_to_collect_inner (stop_func=0x81868e0 <GC_never_stop_func>) at alloc.c:429
#12 0x08187d1b in GC_collect_or_expand (needed_blocks=4, ignore_off_page=0, retry=0) at alloc.c:1242
#13 0x0818a64f in GC_alloc_large (lb=14080, k=1, flags=0) at malloc.c:63
#14 0x0818a9c2 in GC_generic_malloc (lb=14076, k=1) at malloc.c:175
#15 0x0818ace7 in GC_core_malloc (lb=14076) at malloc.c:263
#16 0x080df572 in do_gc_malloc (size=0, what=0x6 <Address 0x6 out of bounds>) at gc-malloc.c:100
#17 0x0813a565 in scm_c_make_vector (k=3517, fill=0x304) at vectors.c:408
#18 0x080e9471 in scm_i_rehash (table=0x97f8b10, hash_fn=0x816c1d0 <scm_ihashq>, closure=0x0, func_name=0x854f8bd "scm_hash_fn_create_handle_x") at hashtab.c:344
#19 0x080e9e23 in scm_hash_fn_create_handle_x (table=0x97f8b10, obj=0xc41c638, init=0x4, hash_fn=0x816c1d0 <scm_ihashq>, assoc_fn=0x80c7eb0 <scm_sloppy_assq>, closure=0x0)
    at hashtab.c:748
#20 0x080ea11d in scm_hash_fn_set_x (table=0x97f8b10, obj=0xc41c638, val=0x4, hash_fn=0x816c1d0 <scm_ihashq>, assoc_fn=0x80c7eb0 <scm_sloppy_assq>, closure=0x0)
    at hashtab.c:804
#21 0x080ea2a7 in scm_hashq_set_x (table=0x97f8b10, key=0xc41c638, val=0x4) at hashtab.c:948
#22 0x08108aee in scm_new_port_table_entry (tag=381) at ports.c:688
#23 0x0813207e in scm_mkstrport (pos=0x2, str=0x4, modes=327680, caller=0x855b842 "call-with-output-string") at strports.c:288
#24 0x08132393 in scm_call_with_output_string (proc=0xbbf49e0) at strports.c:427
#25 0x08143d21 in vm_regular_engine (vm=0x9821060, program=0x0, argv=0x1, nargs=0) at vm-i-system.c:855
#26 0x0813c367 in scm_c_vm_run (vm=0x9821060, program=0xa267de0, argv=0x0, nargs=0) at vm.c:768
#27 0x0813c3cd in scm_load_compiled_with_vm (file=0xa267e60) at vm.c:1079
#28 0x08143d21 in vm_regular_engine (vm=0x9821060, program=0x0, argv=0x1, nargs=1) at vm-i-system.c:855
#29 0x0813c367 in scm_c_vm_run (vm=0x9821060, program=0x9868470, argv=0xffe5ec50, nargs=1) at vm.c:768
#30 0x080d9036 in scm_primitive_eval (exp=0xa261938) at eval.c:692
#31 0x080d908d in scm_eval (exp=0xa261938, module_or_state=0x98b4630) at eval.c:726
#32 0x08117ac2 in scm_shell (argc=12, argv=0xffe5ed94) at script.c:439
#33 0x08068c4f in main (argc=12, argv=0xffe5ed94) at fish.cpp:90

--- End Message ---

> --- Begin Message ---
>
>
>
> Subject: 
>
> Re: bug#19180: guile bug#19180: vacuum_weak_hash_table error
>
>
>
>
> Date: 
>
> Tue, 12 Jul 2016 09:32:16 +0200
>
>
>
>
> User-agent: 
>
> Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
>
>
>
>
> Hi,
>
> On Wed 22 Jun 2016 16:55, Andy Wingo <address@hidden> writes:
>
> > On Mon 15 Dec 2014 07:36, Anand Mohanadoss <address@hidden> writes:
> >
> >> Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1
> >> orig_len=2321 n_items=2321
> >
> > I guess printing a warning is not worse than crashing.  I was unable to
> > make this table work in a reliable way in 2.0 without rewriting it, so
> > in 2.2 there's a new implementation with hopefully no bug in this
> > regard.
>
> I changed this assert to a warning and added a comment like this:
>
>           /* The move to BDW-GC with Guile 2.0 introduced some bugs
>              related to weak hash tables, threads, memory usage, and the
>              alloc lock.  We were unable to fix these issues
>              satisfactorily in 2.0 but have addressed them via a rewrite
>              in 2.2.  If you see this message often, you probably want
>              to upgrade to 2.2.  */
>
> Andy
>
>
> --- End Message ---