Re: Support articulations, slurs and breaths in MIDI (issue 26470047)

[ianhulin44]

Hi Devon,
Like some other posters, I have not yet reviewed your changes in detail,
but would like to make some comments on your patch at the design/concept
level.
First of all, many thanks for looking at the midi code - there are too
few people apart from Jan and I think David K who have much idea of what
happens in there.
On 2013/11/15 09:30:11, Devon Schudy wrote:
> Shouldn't Lilypond's MIDI output be as beautiful as its engraving
output?

Hmmm... yes and no.  LilyPond isn't intended to be a sequencer or an
audio patch editor,
and it doesn't do as much on the audio playback front as some of the
WYSIWYG score editors, whether they are open source projects like Denemo
or paid, closed software such as Finale, Sibelius and friends.
For LilyPond the aims are more limited:  audio playback should provide a
means of proof-hearing what you are trying to produce on the printed
output.  If there isn't a means of writing it on a score, you *probably*
won't need to implement it immediately in the midi performer and/or the
audio set of back-ends which contains MIDI.

> This patch is a step toward that: it makes the most common
articulations work in
> MIDI without requiring the use of articulate.ly and a separate \score.


Good.  You'll need small steps here as the code here is pretty heavily
optimised OOPS coding,
most of which is Jan's.  If you're planning on a patch for this maybe
raise an issue for it in the tracker system.
Also, if your aim is to implement the articulate.ly audio effects
without having to use articulate.ly work-round method of using a
parallel \score block that's a good aim, too.
If you can do this, the next goal for this sub-project may be to get the
audio playback to honour the \repeat structures by translating the volta
and tremolo flavours to unfold.  I looked at this briefly but soon
retired from the fray, gibbering. Until there's a way of doing this,
articulate.ly will still be needed.
I think there was some discussion later on in the thread about producing
audio output sounding as if played by machine or humans.  I reckon
"Human" playback is beyond the scope of the LilyPond project, but a more
accurate audio representation of what appears on the output score is
definitely on-piste.
Another performer-type issue -  very definitely separate from this patch
- is transposed audio output.  Best to note the interaction of the audio
output and the \transpose and \transposition commands as a TODO - the
issue is does Lily apply pitch-bends to effect audio transposition to
implement respect a \transposition command and how does this play nicely
when \transpose is used for the printed output?

> It also overlaps notes in slurs, which gives better results on legato
instruments (e.g.
> winds and strings) than articulate.ly's approach of shortening
unslurred notes.
Looks like a good approach.  Are there any negative results for
percussive and plucked instruments when using this approach?

> Since the meanings of articulations vary widely, their performance can
be
> customized or disabled through properties on the articulation (and the
parser is
> modified to make c-. equivalent to c\staccato, so setting its
properties works).
> This is less convenient than it ought to be, especially for \breathe —
is there
> a better way? Currently slurOverlap is a context property because I
couldn't
> find a better place to put it, but maybe the other properties should
also be on
> contexts so they're easier to set.

> This change doesn't interfere with articulate.ly, since the script's
output
> doesn't contain any articulations.

Is co-existence and inter-operation with articulate.ly an overall goal
for this set of patches?

> Problems:

> 1) Length calculations use written durations rather than absolute
time, which is
> not ideal — switching from 2/2 to 2/4 shouldn't affect the sound at
all. I think
> this can be handled by looking up the tempo and time signature.

> 2) Perform-length is not enough to support everything articulate.ly
does — it
> can't express ornaments that add notes (e.g. \mordent) or change
timing (e.g.
> swing). Ideally, articulations would be able to replace notes with
arbitrary
> music, not just change the length. AFAICT this is hard to implement,
because it
> requires feeding the replacements back into the performers without
interfering
> with the ongoing iteration. (It also requires changing how Audio_item
times are
> determined, so they can start at times other than now_mom(), but I
have a
> solution for this.) An easier alternative would be to have the perform
hook
> return a list of notes, but this doesn't cover \fermata, which needs
to change
> the tempo.
Or have the hook scale up the durations of the notes you return by a
factor you tune via a property for all fermatae?

Hope these comments make sense, Devon, (I'm another 'mere' wind player
and beginner pianist).
Cheers,
Ian


https://codereview.appspot.com/26470047/