Re: preliminary GLISS discussions

[David Kastrup]

Han-Wen Nienhuys <address@hidden> writes:

> On Thu, Aug 30, 2012 at 9:21 AM, David Kastrup <address@hidden> wrote:
>>> Is there a complete proposal of what is on the drawing board? Barring
>>> that, is there a list of (perceived) problems in the current
>>> syntax/parser?
>>
>> The possible values for music following a skipped optional argument are
>> constrained to music parseable without lookahead because the decision
>
> I had to let this sink in for a bit, since I had completely missed
> that a patch doing optional music arguments had went in. Had I noticed
> in time, I probably would have complained (and we might have ended up
> in a flamewar).

It was a matter of "what's good for the goose, should be good for the
gander".  The existing hardwired LilyPond syntax used a variety of
optional arguments (partly skipped explicitly using \default, partly
left out like with \relative { c' }).  Bringing the preexisting syntax
within the scope of music functions meant making Scheme programmers stop
being second-class citizens who has to take what they are fed.

It also meant that the details of syntax between commands, a haphazard
connection of randomness, became unified and dependable.

> I have become convinced that optional, unnamed arguments are not a
> happy design decision, in any language. In Lily it's particularly
> problematic, since we don't group function parameters.  It was already
> difficult to understand in
>
>   \func \a \b \c

Sure.  Particularly if \c is \p instead.  Separate function argument, or
post-event on \b?

I have spent many happy hours among my friends shift-reduce and his even
less likeable cousin reduce-reduce.

LilyPond syntax is not Scheme.  It favors the writer's convenience.

[Some examples of more explicit argument syntax elided]

> If the problem is the language being too implicit, or the parser being
> too complex, they could be avenues to pursue.

The language is implicit, the parser is complex.  If those were
_absolute_ criteria, Scheme/Lisp would be the only human-readable way of
formatting structured data and syntax to computers.  In particular,
LilyPond input itself would be written and delimited like Scheme.
Instead, LilyPond has been designed to end up more or less free-flowing
like musicians talking to each other.  In a way, its syntax is the
antithesis to Scheme.  This comes at a price.

In a manner, part of this price has traditionally been "we are the C++
folks, and we define the vocabulary with which you are allowed to talk
to LilyPond".  This ended up with about six different variants of
\footnote music functions, all doing the same but with slightly
different input and different names.  Using optional arguments instead
meant streamlining the syntax into something the user can actually
remember.

> Of course, we have to first agree that the problem to be solved is
> parser complexity, and not lack of conciseness / excess of red tape.

I am simmering down parser complexity in between.  This is ongoing work,
and it improved maintainability again.  Maintainability is not in high
regard in the LilyPond community: if I ask people to document their
code, the typical reaction is that I get ignored, or that they tell me I
should create an issue in the tracker for every line I want commented,
or that I should submit a patch for review myself.

For my own code, I am not fond of stuff that is hard to maintain, and
the parser has, in its current state, potential for more unification and
simplification.  I have probably sometimes erred on the side of backward
compatibility in the past, but much of it _is_ reachable at moderate
cost after a few iterations.

> Over the years, we worked very hard to cut as much syntactical red
> tape as necessary. This has left us with a parser that is hard to
> understand.

And an input format that is not really suitable for inter-application
communication.  Like PostScript, LilyPond is essentially a write-only
language, with the sole reliable reader being a full interpreter.

> All of the examples above look jarring to my eye; btw.

Yup.  And that's the exact reason we are willing to pay the price.

> The basic
> expression in lilypond looks like
>
>   c4 d4
>
> rather than
>
>   c4 , d4
>
> ie. elements of the language have no explicit separators. That implies
> that we have to do lots of lookahead and precedence twiddling to
> determine how to group parts of the language.
>
>> for skipping requires evaluating the music expression first before
>> letting the parser change state or pushing back a synthetic token which,
>> again, is only possible when there is no lookahead token yet.
>>
>> This is stalling things like
>> <URL:http://code.google.com/p/lilypond/issues/detail?id=2067> and leads
>> to behavior surprising to users.
>
> Why should this be in a music function at all? If the user knows
> enough scheme to understand that "port" means file, he would be expert
> enough to write the expression in Scheme anyway?

You expect the user to write the whole music expression to print in
Scheme as well?

Something like

\withOutputToPort #(current-error-port) \displayMusic c4

can't be made to work because \displayMusic has already produced output
by the time it returns a value.

-- 
David Kastrup