Re: GUILE 2/3 and string encoding cost

[Han-Wen Nienhuys]

Both your hunches were correct:

the code below works, but it makes the GC time go from 2s to 5s.


// GUILE v2 uses the Boehm GC. By allocating all memory through
// scm_gc_malloc, we ensure that all our data structures are traced
// for SCM values. As a performance optimization, we could define
// operator new/delete for data structures that contain just atomic
// data
void *
operator new(std::size_t size) {
  if (guile_booted) {
    return scm_gc_malloc(size, "c++ new");
  }
  return malloc(size);
}

void
operator delete(void *p) {
  // Don't call GC_FREE or scm_gc_free. GC_free manipulates the
  // to-be-freed pointer, and we can't guarantee that the pointer
  // actually comes from GC_MALLOC.
}

On Fri, Jan 24, 2020 at 5:54 PM David Kastrup <address@hidden> wrote:
>
> David Kastrup <address@hidden> writes:
>
> > Han-Wen Nienhuys <address@hidden> writes:
> >
> >> On Fri, Jan 24, 2020 at 12:45 PM David Kastrup <address@hidden> wrote:
> >>> >> No.  Since much of LilyPond's data containing SCM values is stored in
> >>> >> STL containers, it would require serious messing with allocators to get
> >>> >> there.
> >>> >
> >>> > Good point! And I'd agree with you, but if I look at the source code,
> >>> > I can't find much STL structures that really do this. There is
> >>> > Grob_array that would have to be revised, and there is a couple of
> >>> > Drul_array<SCM> instances.
> >>>
> >>> There are a few other structures I think.  Basically one would need to
> >>> go through all mark_smob routines and check for loops over arrays or
> >>> other indirections.  You are right that their number seems reasonably
> >>> small.
> >>
> >> We should be able to short circuit all of this by doing
> >>
> >> +#if GUILEV2
> >> +
> >> +void *
> >> +operator new(std::size_t size) {
> >> +  return scm_gc_malloc(size, "c++ new");
> >> +}
> >> +
> >> +void
> >> +operator delete(void *p) {
> >> +  return scm_gc_free(p, 0, "new");
> >> +}
> >> +
> >> +#endif
> >
> > I wouldn't do that since it would affect all allocated memory even
> > outside of LilyPond proper, like for STL bookkeeping and file buffers
> > and whatnot.
>
> And particularly calling scm_gc_malloc before Guile has even been
> started up might be problematic...
>
> >> unfortunately, it crashes somewhere near the start of GUILE initialization.
> >>
> >> (in GUILE2 the above functions directly call GC_MALLOC, GC_FREE, so I
> >> don't understand why this would be a problem.)
> >>
> >> I am actually quite attracted to moving to GUILE 2 and using BGC.
> >> Debugging GC problems (eg. a constructor triggering GC, which then
> >> deletes the smob under construction) were one of most hairy things to
> >> get right.
> >
> > Them not being done right is the reason for the existence of the
> > Pre_init class...
>
> --
> David Kastrup



-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen