[Lilypond-auto] [LilyIssues-auto] [testlilyissues:issues] Re: #5797 Do not run GC after processing every file.

[Auto mailings of changes to Lily Issues via Testlilyissues-auto]

On Wed, Feb 26, 2020 at 7:39 PM David Kastrup
address@hidden wrote:

It seems strange to measure the effect of the original patch in terms of saving runtime when its obvious motivation was reducing memory usage by collecting memory when the chance of false positives in the mark phase was lowest. So the interesting metric here would be total memory use rather than CPU time, particularly on computers with large memory reserves.

Also this strategy will very likely be significantly more effective on 32bit systems where the danger of false positives in the mark phase is quite higher.

You say it was obvious what this was for, and that was what I assumed
too, but git history says that the (gc) calls were introduced in
46c2a12 which was motivated as

Oops: always do GC around file. Always do debug-gc-assert-parsed-dead check.

ie. it had to do with our detection of objects that leak out of the
per-file context. (Does this mechanism still work with GUILE 2.x?)

I also can't remember doing any kind of measurement about performance,
neither for memory nor for walltime.

It's also only useful in a case where you call lilypond with multiple
files. If there is a concern about imperfect GC for large scores, you
could do

lilypond file1
lilypond file2

iso.

lilypond file1 file2

This is impractical for lilypond-book documents, but it's exactly
those that tend to have smaller scores.

I think it is reasonable to assume that folks who were getting close
to the limit of a 4G memory would have migrated to 64-bit. Practically
speaking, I haven't had a 32-bit system for about 8 years; how would
we go about testing this?

---

[issues:#5797] Do not run GC after processing every file.

Status: Started
Created: Wed Feb 26, 2020 09:39 AM UTC by Han-Wen Nienhuys
Last Updated: Wed Feb 26, 2020 06:39 PM UTC
Owner: Han-Wen Nienhuys

When processing multiple files, we ran GC between the files. The
motivation was that we use a conservative garbage collector, so we get
the most accurate scan when our stack is small. We never measured the
impact of this idea.

On every GC, we pay the cost of tracing through the live set,
ie. fixed data that spans multiple files, and by forcing GC calls, we
trace that data more often than necessary.

We get CPU time savings of ~20 % by not forcing GC.

For GUILE 1.8, the following timings for running "make test-baseline"
on my development branch

force GC on every file

real 3m23.886s
user 8m6.921s

let the GC decide when to collect

real 3m10.523s
user 6m36.925s

For GUILE 2.2, the difference is similar.

https://codereview.appspot.com/579330043

---

Sent from sourceforge.net because address@hidden is subscribed to https://sourceforge.net/p/testlilyissues/issues/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/testlilyissues/admin/issues/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

_______________________________________________
Testlilyissues-auto mailing list
address@hidden
https://lists.sourceforge.net/lists/listinfo/testlilyissues-auto