bug#19883: Smob's mark_smob has become unreliable in Guile 2.x

[David Kastrup]

Mark H Weaver <address@hidden> writes:

> Finalizers are a huge can of worms.  I suspect this is why Ludovic was
> encouraging you to avoid them entirely, but I agree that we need to
> find a way to make them work for when they are needed.

Again, LilyPond does not really have an option to avoid finalizers
(free_smob action): the C++ class instances associated with smobs
contain STL containers, and the storage for those is allocated
separately on the heap.  Running the destructor of the C++ class
instance (which is the basic finalization action done when calling the
delete operator on the C++ class instance) will then also destruct the
STL container, freeing all memory associated with _that_.

We don't have control over the storage layout of the STL templates.
While one could allocate them using a separate allocation template
parameter, that may be a C++11 feature.  And it would really complicate
the code.

Moving all of the smobification stuff into a few template classes has
actually locked up our release process for almost a year: 2.19.15 was
released in September, 2.19.16 is currently being uploaded.  The
release-building crosscompiling environment had to be updated to a newer
GCC version to accept the templated code, and that GCC reason had new
dependencies and needed to be bootstrapped with a C++ rather than a C
compiler.  And the person who has run the release process the last few
years was not actually a programmer.

> The mark functions can ensure that in your Family class, the children
> will be marked even though the parent->child pointers point directly
> to the C++ object.  However, whenever the root of the tree is not
> protected by 'scm_gc_protect_object', the SCM value of that root must
> be visible to the GC, in order to cause the user-specified 'mark'
> function to be called in the first place.
>
> It should be noted that for any particular C++ class, another
> alternative is to arrange for instances of that class to be allocated
> using 'scm_gc_malloc', as Ludovic suggested.  If you do that, then it
> would suffice to have a GC-visible pointer to instances of that class,
> which might make your life easier.

Again: a lot of smobified C++ structures contain STL containers that
have their separate allocation that can easily have pointers to C++
structures associated with smobs again that need to be marked.

At any rate: since LilyPond can eat a significant ratio of the entire
memory space on 32bit machines, I consider it prudent not to have
basically all of that scanned by the conservative garbage collector
since that would significantly increase the number of false positives
when marking.

And we are talking about an advertised GUILE feature after all.  With
regard to "making our life easier": it's a sunk cost since all the work
has obviously been done for GUILEv1 already.

-- 
David Kastrup