Re: the new gc asserts in master

[Han-Wen Nienhuys]

Andy Wingo escreveu:
>> I think reference counting is the correct solution for this, as far as
>> I understand the problem from the quoted message.
> 
> I don't think so; the use case is that (1) we don't want to prevent the
> C object from being freed, so we don't want to hold a reference on the C
> object; but (2) we do want to know when it is freed, so we can release
> our cache; but (3) we want to get the scheme object back if the object
> has not in fact been swept.

I don't understand how reference counting would prevent you from doing
this. Reference counting only is broken if you have (lots of) cyclical
references.  Do you have those?
 
With ref-counting you would loose the unique C object <-> SCM object
mapping, since you could create another reference (another SCM object)
for a given C object if a SCM reference was lost, but a new one
created in the same GC cycle.  From what I've read, this should not be
a problem.  Are you relying on the SCM representation of the C object
being unique?  If not, there should not be a problem.

> But the laziness of the sweeper prevents us from knowing whether the
> cache that we have is in fact accessible, because there is a time
> between the mark phase (in which the object might become sweepable) and
> when the smob's free function is called in the sweep phase (which would
> invalidate the cache).

>> The use of scm_gc_mark() outside of GC is fundamentally broken, since it
>> creates race conditions in the presence of threads.
> 
> I was not aware that this was the case. 
> 
> My impression was that the mark phase is global; it requires all threads
> that were in guile mode to go dormant, and those that were not in guile
> mode cannot enter guile mode until the mark is complete.

Yes, the mark phase is global, but the thread locking is done in
scm_i_gc; once the marking starts, there is only one thread.  Since
scm_gc_mark is called from the smob mark functions, it does not force
other threads to go dormant.  It could, but I suspect the lock would
be a contention point.

If you call scm_gc_mark() on your own schedule, you're venturing
outside the established model.  From what I understand from your
description, you detect that an object is live outside of the mark
phase, which is what triggers the assertion, and which does not force
thread correctness.

> So if I have a thread in guile mode, it is not in the mark phase, hence
> no race. Also, it would not be sweeping; I can check the cache and
> retrieve and mark the object without the thread of interest doing a
> sweep(). But perhaps some other thread would sweep that card, in which
> case I guess I can see where the problem would come in.

An alternative is to have a GC hook, like struct does. It does tie you
to the GC implementation.  I think something that works regardless of
the GC details is better for a user of GUILE.

> It's very irksome that I missed this bit in the documentation.
> 
> So, my proposal to fix this is to expose the sweep mutex as part of the
> API somehow, perhaps as e.g.
> 
>    void* scm_with_sweep_mutex (void* (*with_mutex_func)(void*), void*);
> 
> or so. How do you feel about this? I know it constrains your GC
> implementation, but threads + lazy sweeping + integrating with C
> libraries = exposing some minimal amount of low-level details.

I think it is best to expose as little of the GC implementation as
possible.  The fact that there is a mutex already gives away
implementation details.

I'm a bit miffed that the current interface already gives away that we
won't rewrite (ie. compact) the current heap. Let's not paint
ourselves in more corners.

-- 
 Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen