Re: music macros?

[Erik Sandberg]

[warning: I'm not in a good mood right now. Sorry if I express things in a bad 
way]
On Saturday 04 February 2006 14.43, Han-Wen Nienhuys wrote:
> Nicolas Sceaux wrote:
> > When using raw sexpr instead of music objects, you lose semantics.
> > Making the \relative macro deal with the sexpr:
> >   (make-music 'NoteEvent 'origin <location object> 'pitch <pitch object>
> >   ...)
> > instead of an actual NoteEvent object is a step backward in
> > expressiveness.
> >
> > Wait a minute, I should not be the one argumenting against a LilyPond
> > code -> scheme code compiler!!
> >
>
> But I agree - I think that the parser should handle music expression
> arities, and assemble the right tree structures. I have the feeling
> we'll get into hairiness if we don't, and we would loose error handling
> and recovery that is now provided by bison. This means that we have to
> do the identifier lookup directly, otherwise we can't discover the arity
>   of a music function, and generate the right tree.  

This is the biggest drawback with the approach -- the final step in the 
generation of the "AST" needs to be done at "run-time" (i.e. not at 
"parse-time"). However, probably ~20 lines of scheme code are needed to 
handle this.

While my idea about the structure of the "AST" (output of parser) might have 
looked dirty at first glance, it can be made considerably cleaner. It could 
e.g. be built using a quasi-grammar like this:

element => list | event | identifier | scm
list => (element1 element2 ... ) // Scheme list.
event => Music object
identifier => Scheme symbol // variable/function name. With right number of 
parameters, it should return a Music object
scm => control structure around Scheme value // #()

The parser would spit out a list (according to the above ruleset), which would 
be evaluated by the evaluation function. The evaluation would return the 
desired Music object.

Example. The following:
{ c d \partcombine e <<f g>> }
is converted by parser to:
(seq (c d partcombine e sim (f g)))

c,d,e,f,g are shorthands for real music objects; seq, sim and partcombine are 
Scheme functions. seq and sim are unary functions, which basically are 
curried make-music calls, e.g.:
seq = (lambda (x) (make-music 'SequentialMusic 'elements x))

After parsing, the expression is "evaluated" by a function that simply reads 
everything left-to-right, and as soon as a function is encountered in a list, 
its arity is inspected, after which the right number of parameters are 
evaluated, and the function is finally applied on those parameters. Macros 
get the parameters directly instead, which means that they operate on the 
'element' type described above. I'd say that this is about as expressive as 
real Music trees.

The biggest problem that I see, is that arities are computed at 'runtime', 
which may result in strange things if a macro is applied on an expression 
that contains a not-yet-defined function.

Hm.. and there's a problem with
\lyricsto \new Lyrics { foo bar }
where the parser needs to know when to override/revert the lexical mode. I 
don't know how to handle this right now, my suggestion might be non-feasible 
because of this (but it's late now & I'm sleepy. I'll think more about that 
tomorrow)

>   What we want to 
> postpone is the application of said music function (ie. running the
> appropriate Scheme fragment).
>
> > More seriously.
> >   \relativeMacro c'' { c \someUnaryFunction { d \someMusicVariable } }
> > The parser would read the macro's arguments as unexpanded music objects:
> >
> > (relativeMacro
> >   <Unexpanded music object for "c''">
> >   <Unexpanded music object for the 2nd argument>)

We have a problem with pitches, which is difficult to handle: The c' in 
\relative c' is not the same as in {d'8 c'}. I think my proposal _can_ handle 
this problem, by introducing implicit type conversion (which however may be 
considered a bit dirty).

BTW, implicit type conversion would also be handy for {...}: Sometimes {...} 
denotes a list of music objects, and sometimes it denotes a SequentialMusic 
object. {...} should IMHO be parsed into a Music list, which is implicitly 
converted into a SequentialMusic object if that's requested. This would make 
it possible to soft-code \simultaneous cleanly.

> > Where the two unexpanded music objects looks like:
> >
> >  <EventChord elements: (<NoteEvent duration: <duration 2 0 1 1>
> >                                    pitch: <pitch 1 0 0>)>
>
> just as an aside, changing relative to a macro has some compatibility
> caveats: currently we allow both
>
>    \relative { STUFF }
>
> and
>
>    \relative c'' { STUFF }
>
> if we make \relative generic, then this is no longer possible.

Well, it's quite easy to separate this into two macros:
\rel {STUFF}
and
\relative c'' {STUFF}
this change should be rather convert-ly-safe also.

(I personally dislike when functions/macros have a variable number of 
parameters, and there are no parentheses that clarify the issue)

> > I'm not sure when the #(scheme forms) found in music expressions would
> > be evaluated, maybe when they are read by the parser (iso when the top
> > level music handler expand music objects)...
>
> As #( .. ) is handled in the lexer, not the parser, I think they will
> have to be done directly.

Why not let internal_ly_parse_scm return the un-evalled expression, and 
delaying all evaluations to after parsing?

-- 
Erik