lilypond-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: State of LilyPond with Guile 2.2


From: Jonas Hahnfeld
Subject: Re: State of LilyPond with Guile 2.2
Date: Sun, 11 Apr 2021 19:37:05 +0200
User-agent: Evolution 3.38.4

Am Sonntag, dem 11.04.2021 um 18:04 +0200 schrieb Han-Wen Nienhuys:
> On Sun, Apr 11, 2021 at 4:24 PM Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:
> > 
> > Am Sonntag, dem 11.04.2021 um 16:04 +0200 schrieb Han-Wen Nienhuys:
> > > On Sun, Apr 11, 2021 at 3:42 PM Jonas Hahnfeld via Discussions on
> > > LilyPond development <lilypond-devel@gnu.org> wrote:
> > > > 
> > > > The first set of experiments is about things that developers care
> > > > about, namely 'make test' and 'make doc'. For all runs, I used the
> > > > options "-j4 CPU_COUNT=4".
> > > > 
> > > > 'make test' when compiled with Guile 1.8.8:
> > > >  * User time (seconds): 520.68
> > > >  * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54
> > > > 
> > > 
> > > IIRC, on my 4 CPU machine, the actual time running the regression .ly
> > > files is fairly low (about 1 minute). A lot of time is spent running
> > > the lilypond-book regression tests, which makes this difference all
> > > the more dramatic
> > 
> > True, but the current way of testing lilypond-book is kind of the
> > worst-case scenario for LilyPond because all files are compiled
> > separately. So 37 times a startup time of 2 seconds is already the
> > majority of the slowdown.
> 
> This may warrant some further investigation. Ideally, the shared
> snippet database makes it so we have to run lilypond only a few times.

Yes, this reminds me that I had an idea how to improve those tests...

> 
> > > >  * Maximum resident set size (kbytes): 372560
> > > > 'make test' when compiled with Guile 2.2.6:
> > > >  * User time (seconds): 710.35
> > > >  * Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04
> > > >  * Maximum resident set size (kbytes): 372344
> > > > 
> > > > -> 40% slower when using Guile 2.2.6 during development
> > > > 
> > > 
> > > 
> > > I didn't see you mention this explicitly: for apples to apples
> > > comparisons, you have to disable 'debug-gc-object-lifetimes feature.
> > > It causes a significant slowdown with Guile 1.8 for make test/make
> > > doc, so the comparison is probably worse.
> > > 
> > > This comes at the expense of increased memory usage (which reduces GC
> > > overhead), so you'd have to test with my GUILE patches for a
> > > completely honest comparison. I think the advantage of the decreased
> > > GC overhead is minor, though.
> > 
> > Right, I didn't think about this. However, I compiled without --enable-
> > checking so the actual difference is just garbage collection happening
> > between the files, which you think should be minor. Also "apples to
> > apples" comparison means without your patches, because that's what
> > users get.
> 
> That's not what I meant. The GC between the files is the expensive
> part, because it is done unconditionally and has to mark all the
> reachable data, and sweep the entire heap.  Dropping that (according
> to commit e36b7c7ea98) is a 20% speedup, which we already collected
> for GUILE 2.2, but not for 1.8.
> 
> Due to the heap scaling bugs, removing the GC between files increases
> the heap size with 1.8, and if you don't force GC, for a heap that is
> 5x larger, you run GC 5x fewer times. The GC runs themselves involve a
> larger heap, so this evens out, but you have to mark the reachable set
> 5x fewer times. This is what I meant with a 'minor' advantage.

So here are the numbers with debug-gc-object-lifetimes set to #f:
'make test':
 * User time (seconds): 397.05
 * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.10
 * Maximum resident set size (kbytes): 897048

-> a 15% improvement over the default, at roughly 2.5x the heap times 4
processes (I think, not sure how exactly /usr/bin/time works)

'make doc':
 * User time (seconds): 2557.86
 * Elapsed (wall clock) time (h:mm:ss or m:ss): 16:23.00
 * Maximum resident set size (kbytes): 1004776

-> roughly 10% at 2.1x the heap


> > > What does "switch to" mean precisely? Do you mean "remove support for
> > > guile 1.8", "make 2.2 the default (when given the choice) but keep
> > > supporting 1.8" ?
> > 
> > Eventually, I think the target should be to "remove support for Guile
> > 1.8". In particular once we provide binaries with Guile 2.2, that will
> > be the one version we care about.
> 
> From an engineering perspective, I like the idea of dropping 1.8,
> because we can then simplify the C++ GC marking glue which is hard to
> get right, but otherwise, it's still not very compelling.
> Functionally, the only thing that seems really relevant is Unicode
> string handling on the Scheme side.
> 
> I wonder if it isn't more practical to fork guile 1.8, and stick it
> into our tree as a submodule, and always build lilypond against the
> in-tree version. We'd be up for the maintenance on the 1.8 branch, but
> it might well be less work than keeping up with the churn that newer
> GUILE versions bring us.
> 

I had already replied that I don't like that option; it was always a
given for me that LilyPond would move on. Guile 2.2 also makes binary
distribution much nicer (because there no more shared srfi libraries,
so lilypond can be linked as one static executable) and makes it
possible to offer 64 bit executables for Windows.

But given the reactions, I'll reduce activity on my work towards Guile
2.2...

Jonas

Attachment: signature.asc
Description: This is a digitally signed message part


reply via email to

[Prev in Thread] Current Thread [Next in Thread]