Re: Don't reload initialization files when processing multiple files (issue 5874044)

[dak]

Reviewers: Graham Percival,

Message:
On 2012/03/22 03:43:47, Graham Percival wrote:
> This feels like a dangerous change.  Obviously we don't /want/
"bleed-through"
> of parameters or memory from one file to the next, but I would be
shocked if we
> don't have any of that right now.

Don't overestimate the issue.  What is involved here is bleedthrough
from init.ly, namely from one parser incarnation to the next.  Any
bleedthrough in scm is left unchanged.

It turns out that
git grep hash ly
appears surprisingly sufficient for diagnosing the bleedthrough we are
actually seeing.

Bleedthrough is from destructively changed data.  That means hashtables
need to be either unchanged by user code, or reset each time.  The same
holds for everything else of course, like destructively modified alists
and output definitions and what else.

But there is a rather surprising absence of those.

> We'll just get reports dribbling in over a few weeks.

I doubt it.  The only situation where bugs can show is when multiple
files are being compiled at once.  That is not the usual command line
usage.  So reports can be basically about

a) documentation differences
b) problems with lilypond-book documents

I doubt we'll get to see anything outside of lilypond-book usage.  Note
that we already have bleedthrough potential in .scm (which we apparently
managed to keep under control).

My original plan was to split the various init files into parts that
need to be rerun every time, and parts that can be preloaded once.

But it turned out that the "need to be rerun" stuff basically concerned
output definitions, and cloning them looked feasible.

Description:
Don't reload initialization files when processing multiple files

There should probably be something approaching an option for that.  It
should mostly speed up the regtests.  It helps "make doc" less than
expected, but probably still some.

Please review this at http://codereview.appspot.com/5874044/

Affected files:
  ly/init.ly
  scm/lily.scm


Index: ly/init.ly
diff --git a/ly/init.ly b/ly/init.ly
index  
6546ed422aa676bffd633bacf013b9c8e7086e9b..7a49103b476208a48bd7e2931827b343da286465  
100644
--- a/ly/init.ly
+++ b/ly/init.ly
@@ -6,8 +6,42 @@

 \version "2.15.18"

-\include "declarations-init.ly"
+#(if (not (pair? lilypond-declarations))
+     (ly:parser-include-string parser
+                              "\\include \"declarations-init.ly\""))

+%% We need to save the variables of the current module along with
+%% their values: functions defined in the module might refer to the
+%% variables
+
+#(if lilypond-declarations
+     (if (pair? lilypond-declarations)
+        (begin
+          (for-each
+           (lambda (p)
+             (let ((var (cadr p))
+                   (val (cddr p)))
+               (variable-set! var
+                              (if (ly:output-def? val)
+                                  (ly:output-def-clone val)
+                                  val))
+               (module-add! (current-module) (car p) var)))
+           lilypond-declarations)
+          (note-names-language parser default-language)
+          (use-modules (srfi srfi-1)))
+        (module-for-each
+         (lambda (s v)
+           (let ((val (variable-ref v)))
+             (if (not (ly:lily-parser? val))
+                 (set! lilypond-declarations
+                       (cons
+                        (cons*
+                         s v
+                         (if (ly:output-def? val)
+                             (ly:output-def-clone val)
+                             val))
+                        lilypond-declarations)))))
+         (current-module))))

 #(ly:set-option 'old-relative #f)
 #(define toplevel-scores (list))
Index: scm/lily.scm
diff --git a/scm/lily.scm b/scm/lily.scm
index  
0c20c148b20a5ec56066379eaee887eeab20bf8b..64c403b0749432e55df286cae181876a3a1cebde  
100644
--- a/scm/lily.scm
+++ b/scm/lily.scm
@@ -845,6 +845,8 @@ PIDs or the number of the process."
         (dump-profile "lily-run-total" '(0 0) (profile-measurements)))
     failed))

+(define-public lilypond-declarations '())
+
 (define (lilypond-file handler file-name)
   (catch 'ly-file-failed
          (lambda () (ly:parse-file file-name))