Re: GUILE 2/3 and string encoding cost

[Hans Åberg]

> On 24 Jan 2020, at 10:03, Han-Wen Nienhuys <address@hidden> wrote:
> 
> On Thu, Jan 23, 2020 at 10:39 PM David Kastrup <address@hidden> wrote:
> 
>> 
>>> on mozart-hrn-3, over 3 runs, we get
>>> 
>>> 2.0.14  - avg 2.1s
>>> 1.8.8 - avg 0.31s
>>> 
>>> so the new GC is about 5-10x slower than the old one. With GUILE 1.8,
>>> garbage collection covers typically is 10% of the runtime, so all things
>>> equal, the Boehm GC would cause a 1.5-2.0x slowdown in the total.
>>> 
>>> It would be good to see how the JITting of code impacts Scheme
>>> execution.
>> 
>> Boehm GC can work in a background thread I think.  And Guile-v2
>> applications typically just let all their data be treated as pointers
>> rather than using a smob-marking algorithm like we do, and it is
>> conceivable that Boehm GC's individual mark function does not scale.
>> 
> 
> Do you mean our mechanism to call user-defined mark functions? I doubt that
> there are obvious BGC scalability problems in BGC's mark functoin.

The Boehm GC collects in separate threads, however, when the program is 
threaded, it puts locks around every allocation, which is very time consuming.

> However, considering everything a pointer for a 32bit application that
>> can eat a significant ratio of the total address space is a nightmare:
>> there would be just too much memory pinned down due to conservative
>> garbage collection.
>> 
>> 
> GUILE 1.8 already scanned the stack conservatively, so large scores would
> probably never work on 32 bits. Was this a concern in the past?  How do
> score sizes (in pages) translates to memory usage (in megabytes)?
> 
> I think it is reasonable for us to start assuming people run lilypond on a
> 64-bit machines.
> 
> 
>> On a 64bit application, this would be somewhat more tenable, but we'd
>> need to override operator new for smobs.
>> 
>> Or do we?  Maybe the heap is collected by default, and we need to switch
>> that off?
>> 
> What do you mean with "heap is collected”?

The Boehm GC also has a malloc/free pair that keeps track of pointers into the 
GC heap. One can use them to define global operator new/delete, which the C++ 
standard guarantees that the linker will use them for all allocations. This can 
combined with GC allocations.