Re: [PATCH] Marking weak alist vectors

[Ludovic Courtès]

Hi Marius,

Marius Vollmer <address@hidden> writes:

> Yes, SMOBs could profit from a guarantee about the order of
> finalization, but I don't think that guarantee can be implemented
> cheaply enough.  (Or can it?)
>
> So, my current point of view is that smobs have to suffer in order to
> make live easier for the GC.

IMO, it's not just a fancy improvement for SMOBs but a requirement.
That is, if we cannot provide this guarantee, then we have to either
completely remove SMOBs (hmm, not sure this is a good idea ;-)) or
disallow their use in weak vectors.  Again, for the SMOB programmer
viewpoint, the order of "finalization" is something that should be taken
seriously.

>> SMOBs shouldn't ever have the impression that a value attached to a weak
>> key is freed _before_ that weak key.
>
> This would not only apply to weak keys etc, but to all kinds of
> references, right?  I.e., all SCM values belonging to a smob would be
> guaranteed to be valid when the free function for that smob is called.
> Thus, we are not only talking about your patch for weak hash tables,
> but about big changes to the GC in general, right?

Well, maybe you're extrapolating too much.  ;-)

Of course, this issue is very similar to the one I raised initially.
However, a SMOB that embeds SCM values knows that it is responsible for
marking them.  Therefore, when a SMOB is freed (because it hasn't been
marked), it knows that the SCM values under its control may have
remained unmarked as well and may have been GC'd.

So I think the semantics here are different: you cannot reasonably
expect SCM values embedded within a SMOB to be freed after the SMOB.

> Heh, that is not an example, that is a test case. :-)

Yeah, but it's pretty close to an example.  ;-)

>> For a more general example where this is an issue:
>> http://lists.nongnu.org/archive/html/g-wrap-dev/2005-09/msg00006.html
>> (see at the bottom).
>
> (I only read the article you linked to directly, not the thread.)
>
> So you have a C API that requires a certain order for freeing objects
> and that order needs to be honored by the smobs that wrap the objects,
> right?

Right.

> My immediate reaction is to suggest to fix this by layering something
> on top of the C API, such as the reference counting that you mention
> in your article.  (Refcounting is acceptable since cycles must be
> prevented anyway.)

Sure, reference counting is the usual way to solve that problem (for
instance, that's how glib, gnet, etc. solve this).  But I was convinced
that using object properties would achieve the same goal much more
easily.

> But note that the thing we are talking about now is very different
> from what you started out with: now we are not talking about in which
> run of the GC a value belonging to a weak key is collected (i.e., the
> key in run n, the value in run n+1), but in what order objects are
> finalized within a single run of the GC (if I am not confused now).

As stated earlier, I think your attempt to generalize the problem is
unfruitful.  ;-)  I.e., IMO, the general issue of the order of
finalization within a single run of the GC isn't an actual issue.
We can really tackle the first problem (i.e., "in which run a value
belonging to the GC is collected") without having to worry about the
order of finalization in general.

Furthermore, I think the patch shows that the first issue can be fixed
without introducing a significant performance cost.  The second version
of the patch introduces _some_ overhead in `scm_i_remove_weaks ()',
because it needs to iterate on MARK_QUEUE, but that doesn't seem to be
too much of a burden.

In [0], this issue is not even mentioned.  In fact, I believe it is
quite specific to Guile since other Scheme implementations maybe don't
have such a large C API.  Further research suggests that this is also an
issue in languages such as SmallTalk where finalization is visible to
the application [1,2].  The Java specs (almost) clearly state that a
weak reference is only finalized after the object it refers to has
become "weakly reachable" (well, in the end it says nothing about the
finalization of the object itself...) [3].

Thanks,
Ludovic.

[0] http://www.haible.de/bruno/papers/cs/weak/WeakDatastructures-writeup.html
[1] http://mail.python.org/pipermail/python-dev/2003-November/040299.html
[2] 
http://www.mimuw.edu.pl/~sl/teaching/00_01/Delfin_EC/Overviews/WeakReferencesAndFinalization.htm
[3] http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ref/WeakReference.html