Re: 2.16 release candidate 3 imminent

[David Kastrup]

"address@hidden" <address@hidden> writes:

> On Jan 21, 2012, at 10:15 PM, David Kastrup wrote:
>
>> "address@hidden" <address@hidden> writes:
>> 
>>> On Jan 21, 2012, at 7:58 PM, David Kastrup wrote:
>> 
>>> 
>>>    that all articulation events will be pulled out of NoteEvents or
>>> 
>>>    RestEvents and broadcast at the iterator level.
>>> 
>>> 
>>>    There is such a thing as a chord articulation.
>>> 
>>> 
>>> 
>>> Why couldn't this distinction be captured via a different event name?
>>> ChordArticulationEvent versus ArticulationEvent, for example.
>> 
>> Where would be the point?
>
> To not make the distinction in the parser.

It is different input anyway, so the distinction is already there in the
input.  If you would drop that distinction in the rules that early, you
would end up with chords being allowed as chord constituents.

> I think that changing the parser should be a last resort if we can't
> figure out a way to represent information through different event
> classes.

I have trouble understanding what you call "parser" here.

>>> What would really help are some before/after examples (ly code and/or
>>> music streams and/or brief text like "before the patch, you could not
>>> do X, after, you can" or "this patch will allow me to experiment with
>>> implementing X") would help a great deal.  As if it were going into
>>> the Change Log, for example.
>> 
>> It's a bit hard since the whole design (perfected by the
>> rhythmic-music-event) was intended to make no user-visible difference.
>> The music expression has just become predictable.  You get an EventChord
>> iff < ... > has been in the input.  You get articulations on NoteEvent
>> for pitch-postevent regardless whether or not this is part of a chord.
>> 
>
> Why can't we treat c as shorthand for <c>?

Because inside of chords, it isn't a shorthand for <c>.  And music
functions like \tweak need to work inside of chords as well (\tweak
currently even _fails_ if you apply it to c where LilyPond considers it
a shorthand for <c> and the corresponding passage in the NR is _really_
embarrassing to LilyPond as a system).  See
<URL:http://code.google.com/p/lilypond/issues/detail?id=2037#c3> for an
approach to make c as <c> work which ultimately went up in flames.  It
is not like I did not seriously try that approach.  It is a total
maintenance nightmare (and has logical fallacies you can't really iron
out) unless you forbid music functions inside of chords altogether.

It also makes user level programming tasks at the input side (and that's
really what music functions are) much harder and more intransparent if
you don't have predictable relations between input and music
expressions, and if you have to strip multiple layers before getting at
the information you actually wanted and that was straightforward in the
input.

Music expressions _represent_ the input, as opposed to stream events
which represent the typesetting task.

> Having a compulsory wrapping around singletons is just as predictable
> as no wrapping at all.

Except that inside of chords, there is no such wrapping, and a music
function does not know in advance where it is going to get used.

> As for articulations being on a NoteEvent whether or not this is part
> of a chord, I agree that this is a very good idea, but I think this
> can be accomplished by wrapping everything in an event chord and
> farming out the work to iterators.

Uh, we _are_ farming this out to the rhythmic event iterator.  That was
its point.  It was the missing link.

>> If you use \displayMusic on something that you might want to put into a
>> chord, you don't get wrong input.  Tacking < > around a construct does
>> not change the structure of its inside.  It is not necessary to tack < >
>> around a construct to make a \tweak work (which is a user-visible
>> change). 
>
> Why couldn't tweak make this distinction in the music function itself?

Why should it?  Why should we strive to make programming a reliably
working music function as hard as possible?

> The music function could check if its input is an EventChord and, if
> so, feed it the first entry in the 'elements list of the EventChord.
> This seems like a less intrusive change than changing the parser.

And what if there is more than one element?

Anyway, see above for an attempt to make a less intrusive change and get
inside chord functions work with the same argument parsing semantics as
normal music functions (after all, one does not know the difference).
It failed.  A special inside-of-chord-mode for parsing music function
arguments is a total maintenance hog.  Nobody will understand the code,
and more importantly, nobody will understand the behavior.

I have been deadlocked for about a month on further parser work by this.

>> You can use #{ #} for constructing material that ends up _inside_ of a
>> chord.  Something like
>> \displayLilyMusic < \displayLilyMusic ##{ c-. #}>
>> does not go up in flames but just does what one would expect.
>
> This is indeed an interesting case.  Again, though, I would expect
> this to fail, precisely because #{#} manipulates the material.

#{ #} does not manipulate material.  It takes what the parser considers
"music" or a number of other constructs.  That means that currently you
can't use it to provide the return value for a music function called
inside of a chord.

And you currently _can't_, I repeat _can't_ write a music function
(unless it gets a music argument created by the parser instead of just
passed in from elsewhere) that can work like the input "c" inside of a
chord as well as outside.  Because outside, you need to produce <c>, and
inside, you need to produce c.  But without a music argument?  No.
Writing a music function called as \semitonesabovec #5 (or something
like that) that will work both inside as well as outside of chords is
just _impossible_ right now.  It will be trivial afterwards.

> If we define c as shorthand for <c>, then this would be like doing
> \displayLilyMusic < \displayLilyMusic <c> >, which I would also expect
> to fail.

Which is just one reason why defining c as shorthard for <c> in the
front end is a bad idea.

Currently, the parser allows you to specify music arguments and Scheme
arguments _interchangeably_.  The parser has no convincing case meddling
with Scheme expressions.  A music function argument derived in chord
context can't be passed on into music list context and vice versa if you
need to add and strip arbitrary wrappers.  I thought differently at one
time, but my attempts to make this work, as witnessed by
<URL:http://code.google.com/p/lilypond/issues/detail?id=2037#c3>, went
up in flames.

And I am good at this.  Real damn good.  I was not comfortable with
requiring this level of goodness for maintenance, but I thought few
others would need to touch it.

If nothing like that is in the parser, naturally nobody will need to
touch it.  Or even think about it.

The best maintainable code is code that is not there.

> I'm playing devil's advocate because it is an important change and I
> want to make sure that its benefit outweighs the cost of adding
> exceptions into the code base that facilitate the in/out of EventChord
> distinctions.

You had to do that anyway if you were serious about it because c-. was
never the same as <c-.>.  This is not just a wrapper that goes on and
off, it is a transformation.

For the release, I will add a documented function that does that
transformation, so if you have your own analysis function, you can just
pass its music arguments through that transformer.

It might even be feasible to create a special converting
define-music-function variant (that you could at the start of the
document assign to the standard function) that puts all of its arguments
through the conversion.

This define-music-function variant, however, would _fail_ to work inside
of chords (for the same reason that the above mentioned issue failed).
So it would be a last resort solution for document conversion, and you
should not let it loose on the LilyPond code base itself.

-- 
David Kastrup