T3154 work - question on init code, ly_book_scheme initialization passing Lexer instance as value for scheme (parser), why is this?

[Ian Hulin]

I don't understand why when I set a breakpoint at ly_book_scheme I get a
parameter passed which should be the value of scheme (parser) coming
through as a Lexer type. This is part of work for issue 3154.

But here's a summary of what's I've been doing and some gdb ouptut.

Once upon a time I thought getting alternately named and/or suffixed
would be a nice small-scale fix I could get done and dusted in time to
go into V2.18.  As David K would say, colour me three shades of older,
sadder and wiser.  It's involved changing quite a few files.

The basic design is to track the incoming \midi {} blocks, reference
them in an enhanced Performance:: object and then haul these out and
scan them for file-name="blah" and file-suffix="thingy" clauses before
writing out the midi files in the procedure in scm/midi.scm.  The idea
here is to use output_def_lookup to get hold of the property values.

Unfortunately the only hook available was get_midi in the parser code,
so I had to get this to save to a new parser property current_midi_, and
add a getter and setter method also.  Additionally I had to add a
current_parser_ to the Paper_book:: object with new getter and setter
methods so that the code in paper_book.cc that calls through eventually
to the midi.scm could get hold of the properties.

In order to ensure I got hold of the right parser instance I had to
tweak the scheme procedures called by initialization to pass through
their current parser scheme value into ly:book_process (a.k.a
ly_book_process in C++ code).  The other path through there is
ly:book-process-to-systems/ly_book_process_to_systems which I tweak in a
parallel fashion to ly_book_process.

I've changed the LY_DEFINE definition for ly_book_process to

LY_DEFINE (ly_book_process, "ly:book-process",
             4, 0, 1, (SCM book_smob,
                       SCM default_paper,
                       SCM default_layout,
                       SCM output,
                       SCM parser_ref),

This is so that some user who calls ly:book_process with the old
signature from scheme doesn't get their code broken by this change.

Simple testing with GDB shows that if the procedure is called with just
the four required parameters, the C++ sees parser_ref as being SCM_EOL
(alias '()), and so I should be able to handle this.

I've guarded the LY_ASSERT_SMOB call with

  /*
    UGH - working round that initialization may pass an instance of
    the lexer in parser_ref.
   */
  Lily_parser * my_parser;
  //Lily_lexer * my_lexer;
  my_parser  = unsmob_lily_parser ( parser_ref ); // 0 if parser_ref ==
SCM_EOL
  if (parser_ref != SCM_EOL
      && !parser_ref)
    {
      LY_ASSERT_SMOB (Lily_parser, parser_ref, 4)
    }


However, I get this from an ly:parse-file call in the sandbox with one
of the regression test sources (input/regression/midi/key-initial.ly) as
the input:
I've set a break in gdb at ly_book_process and printed the scheme value
passed in parser_ref (the new, optional parameter)

(gdb) help ps
ps - print variable packed in a SCM structure
Interpret $arg0 as a scheme string
by calling scm_display and scm_newline
to output them to stdout
(gdb) ps parser_ref
(#<Lily_parser #<Lily_lexer (#<module b56588e0>) > >)
$41 = void
(gdb)

Can anyone help explain how/why I get this instead of a simple
#<Lily_parser>, please?

I've put a snap print-out in print-book-with in scm/lily-library.scm to
show how it's been called, it printing
print-book-with parser=#<Lily_parser #<Lily_lexer (#<module b56588e0>) >
> book=#<Book> process-procedure=#<primitive-procedure ly:book-process>

This is the scheme expression being interpreted from further up the
stack at the breakpoint:

(gdb) frame 8
#8  0x0833e860 in Lily_lexer::eval_scm_token (this=0x86324b8,
sval=0xb0c7f5c0)
    at /home/ian/src/lilypond/lily/include/lily-lexer.hh:56
56        SCM eval_scm_token (SCM sval) { return eval_scm (sval, '#'); }
(gdb) ps sval
((let ((book-handler (if (defined? (quote
default-toplevel-book-handler)) default-toplevel-book-handler
toplevel-book-handler))) (cond ((pair? toplevel-bookparts) (let ((book
(ly:make-book $defaultpaper $defaultheader))) (map (lambda (part)
(ly:book-add-bookpart! book part)) (reverse! toplevel-bookparts)) (set!
toplevel-bookparts (list)) (if (pair? toplevel-scores) (map (lambda
(score) (ly:book-add-score! book score)) (reverse! toplevel-scores)))
(set! toplevel-scores (list)) (book-handler parser book))) ((or (pair?
toplevel-scores) output-empty-score-list) (let ((book (apply
ly:make-book $defaultpaper $defaultheader toplevel-scores))) (set!
toplevel-scores (list)) (book-handler parser book))))) . #<location
/home/ian/src/lilypond/build/out/share/lilypond/current/ly/init.ly:53:2>)
$42 = void
(gdb)


Below is the full gdb output up to the breakpoint.
===================================================

Log level set to 287
GNU LilyPond 2.17.22
Relocation: is absolute: argv0=/home/ian/src/lilypond/build/out/bin/lilypond
PATH=/home/ian/src/lilypond/build/out/bin (prepend)
Setting PATH to
/home/ian/src/lilypond/build/out/bin:/home/ian/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
Relocation: compile datadir=, new
datadir=/home/ian/src/lilypond/build/out/share/lilypond//current
Relocation: framework_prefix=/home/ian/src/lilypond/build/out/bin/..
Setting INSTALLER_PREFIX to /home/ian/src/lilypond/build/out/bin/..
PATH=/home/ian/src/lilypond/build/out/bin/../bin (prepend)
Setting PATH to
/home/ian/src/lilypond/build/out/bin/../bin:/home/ian/src/lilypond/build/out/bin:/home/ian/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
Setting GUILE_MIN_YIELD_1 to 65
Setting GUILE_MIN_YIELD_2 to 65
Setting GUILE_MIN_YIELD_MALLOC to 65
Setting GUILE_INIT_SEGMENT_SIZE_1 to 10485760
Setting GUILE_MAX_SEGMENT_SIZE to 104857600

LILYPOND_DATADIR="/usr/local/share/lilypond/2.17.22"
LOCALEDIR="/usr/local/share/locale"

Effective prefix: "/home/ian/src/lilypond/build/out/share/lilypond/current"

Guile/Scheme load path
main.cc,263: %load-path=
        "("/home/ian/src/lilypond/build/out/share/lilypond/current/scm"
        "/home/ian/src/lilypond/build/out/share/lilypond/current"
        "/usr/local/share/guile/site"
        "/usr/local/share/guile/1.8"
        "/usr/local/share/guile"
        )"
PATH="/home/ian/src/lilypond/build/out/bin/../bin:/home/ian/src/lilypond/build/out/bin:/home/ian/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/lily.scm]
Guile 1.8
[[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/lily.scm]
Guile 1.8
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/lily-library.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/output-lib.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/markup-macros.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/parser-ly-from-scheme.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/file-cache.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-event-classes.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-music-callbacks.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-music-types.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-note-names.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/c++.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/chord-entry.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/skyline.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/stencil.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-markup-commands.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/markup.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/modal-transforms.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/chord-generic-names.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/chord-ignatzek-names.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/music-functions.scm
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-music-display-methods.scm]
]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/part-combiner.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/autochange.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-music-properties.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/time-signature-settings.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/auto-beam.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/chord-name.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/bezier-tools.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/ly-syntax-constructors.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-context-properties.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/translation-functions.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/script.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/midi.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/layout-beam.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/parser-clef.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/layout-slur.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/font.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/encoding.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/bar-line.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/flag-styles.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/fret-diagrams.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/tablature.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/harp-pedals.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-woodwind-diagrams.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/display-woodwind-diagrams.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/predefined-fretboards.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-grob-properties.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-grobs.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-grob-interfaces.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/define-stencil-commands.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/scheme-engravers.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/titling.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/text.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/backend-library.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/x11-color.scm]
[/home/ian/src/lilypond/build/out/share/lilypond/current/scm/safe-lily.scm]


Initializing FontConfig...
Adding font directory:
/home/ian/src/lilypond/build/out/share/lilypond/current/fonts/otf
Building font database...

Processing
`/home/ian/src/lilypond/build/out/share/lilypond/current/ly/scheme-sandbox.ly'
Parsing...
[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/init.ly
[<string>
 
[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/declarations-init.ly

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/music-functions-init.ly]
  [/home/ian/src/lilypond/build/out/share/lilypond/current/ly/toc-init.ly]
Using `nederlands' note names...

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/drumpitch-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/chord-modifiers-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/script-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/chord-repetition-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/scale-definitions-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/dynamic-scripts-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/spanners-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/predefined-fretboards-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/string-tunings-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/property-init.ly]
  [/home/ian/src/lilypond/build/out/share/lilypond/current/ly/grace-init.ly]
  [/home/ian/src/lilypond/build/out/share/lilypond/current/ly/midi-init.ly

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/performer-init.ly]]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/paper-defaults-init.ly

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/titling-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/text-replacements.ly]]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/context-mods-init.ly]

[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/engraver-init.ly]]]
Using `nederlands' note names...
 [/home/ian/src/lilypond/build/out/share/lilypond/current/ly/scheme-sandbox.ly
guile> ly:parse-file
#<primitive-procedure ly:parse-file>
guile> (ly:parse-file
"/home/ian/src/lilypond/input/regression/midi/key-initial.ly")

Processing `/home/ian/src/lilypond/input/regression/midi/key-initial.ly'
Parsing...
[/home/ian/src/lilypond/build/out/share/lilypond/current/ly/init.ly
Using `nederlands' note names...
 [/home/ian/src/lilypond/input/regression/midi/key-initial.ly]
print-book-with parser=#<Lily_parser #<Lily_lexer (#<module b56588e0>) >
> book=#<Book> process-procedure=#<primitive-procedure ly:book-process>
Breakpoint 2, ly_book_process (book_smob=0xb0c7ec98,
default_paper=0xb5db5d88, default_layout=0xb5a69170, output=0xb5654e40,
    parser_ref=0xb0c80e60) at /home/ian/src/lilypond/lily/book-scheme.cc:78
Breakpoint 2, ly_book_process (book_smob=0xb0c7ec98,
default_paper=0xb5db5d88, default_layout=0xb5a69170, output=0xb5654e40,
    parser_ref=0xb0c80e60) at /home/ian/src/lilypond/lily/book-scheme.cc:78
78        Book *book = unsmob_book (book_smob);
(gdb) list
63      LY_DEFINE (ly_book_process, "ly:book-process",
64                   4, 0, 1, (SCM book_smob,
65                             SCM default_paper,
66                             SCM default_layout,
67                             SCM output,
68                             SCM parser_ref),
69                 "@var{ly_book_process}: C++ alias for Scheme
ly:book-process"
70                   "Print a book specified by the @var{book_smob}
parameter."
71                   "Use @var{default_paper} and @var{default_layout}
as paper and"
72                   "layout blocks to be used when printing this instance."
73                   "Pass the @var{output} to the backend unchanged. It
holds information"
74                   "defining the output for the backend.  "
75                   "For example, for local, file-based outputs it may
be a string holding the"
76                   "basename for the output file, or it could be a
socket (for network-based output).")
77      {
78        Book *book = unsmob_book (book_smob);
79
80        LY_ASSERT_SMOB (Book, book_smob, 1);
81        LY_ASSERT_SMOB (Output_def, default_paper, 2);
82        LY_ASSERT_SMOB (Output_def, default_layout, 3);
83        /*
84          UGH - working round that initialization may pass an instance of
85          the lexer in parser_ref.
86         */
87        Lily_parser * my_parser;
88        Lily_lexer * my_lexer;
89        my_parser  = unsmob_lily_parser ( parser_ref ); // 0 if
parser_ref == SCM_EOL
90        if (parser_ref != SCM_EOL
91            && parser_ref != 0)
92          {
(gdb) help ps
ps - print variable packed in a SCM structure
Interpret $arg0 as a scheme string
by calling scm_display and scm_newline
to output them to stdout
(gdb) ps parser_ref
(#<Lily_parser #<Lily_lexer (#<module b56588e0>) > >)
$41 = void
(gdb)