Evolution of thought on composing with Lily

[Jeff Henrikson]

Hi again,

I had a school project a month or so ago to compose a jazz arrangement for 5 
wind instruments plus piano, bass, guitar, and drums.
As an experiment and to force myself to learn lily better, I decided to compose 
it straight into a text file, using only a closed
score sketch on paper.  This got really horrible really quickly because I had 
to do things like harmonize four, five or six voices
over chord changes.  But since to extract parts I had to keep them each on 
their own staff, I had to keep them separate in the text
file, which was horrible.

Which led me to reflect on this problem I have had with lily from day one: why 
are you embedding a two dimensional thing like a
musical score in a one dimensional thing like a text file?  After all, 
composers using the thing we know of as music notation have
always written scores in a way that they could work both vertically and 
horizontally for a reason.

My answer is that it seems like most of the music I see on the net done with 
lily is transcriptions of scores which already exist
somehow.  My problem is that I am creating scores that don't exist, and to 
write it out on paper and then copy it into lily
introduces all sorts of extra work and synchronization errors when stuff starts 
getting edited, etc.  As I have mentioned to some
before, I have been thinking about the Music notation GUI problem for several 
years, and am convinced that there is probably 5x
speed increase possible in a good computer GUI over pencil and paper scoring, 
especially on larger scores because there is a lot of
redundancy and because keys are inherently faster than pencils drawing not only 
characters but graphical symbols.  So to write out
completely in pencil and then take at least the same amount of time again to 
enter lilypond is fairly hideous to me.  Again,
entering a fixed piece of music line by line in .ly is clearly faster than 
copying by hand, but not so for things which get changed
and come into existence out of order.

So my first attempt at restoring my own sanity was a .ly file with a lot of 
little sections that looked like this:

%  Bridge                  | ab            | ab   f-7       |
spAa = \notes \relative c' {g2. g2. ~      | g2. ~ g2.  ~   |
spAb = \notes \relative c' {c2. ~ c2.  ~   | c2. ~ c2.  ~   | ... -->
spAc = \notes \relative c' {aes2. ~ aes2.~ | aes2. ~ aes2. ~|
spAd = \notes \relative c' {g2. ~ g2.  ~   | g2. ~ g2.  ~   |


And so on for about 8 bars or so at a time.  Then I would go include those 
symbolic defs in the actual parts to indicate the
overall form.  Now this worked really great except that managing the tabs and 
making sure everything lined up again after making a
change was a major pain in the ass.  I ended up not editing my score well at 
all because it was so laborious and was generally very
unhappy with the result, not only in the printed score, but in the music itself 
because I was so burdened when writing it.

So for my next score, I made three changes of approach.  First, I wrote out the 
score completely in pencil first.  Second, since a
pencil score was good enough, I was only using lily to copy parts for players, 
so I completely omitted the rhythm section as they
are totally useless to put into lily.  Only the 6 wind instruments were 
entered, so that the transpositions could be done by
computer, and so that the copying could be crosschecked for correctness by the 
midi sequencer (as it is possible to sit at a
keyboard and play through a score to correct wrong notes, but not so once parts 
have been extracted by hand.)

Third, and most importantly, I decided to write a .ly file in a spreadsheet.  
MS Excel is what was on my machine, so that was the
implement of choice (sorry free software and otherwise non-MS guys.)  It was 
able to export a tab-delimited text file, and since
tab is whitespace, it was all the same to lily.

I ran into the following difficulties:
- Excel has some braindead quoting behavor when it writes the text file.  Fix: 
a perl regexp substitution that reverses the
quoting.
- Barchecks were annoying to insert by hand.  Fix: a perl regexp substitutiton 
that uses the tab delineation to put barchecks
between things that look like containing notes
- Lily knows how to expand a R1*n into little R1s, but it doesn't know how to 
congeal multiple R1s into a big R1*n, which, along
with rehearsal marks is absolutely necessary when handing parts to human 
players.  Fix: a perl regexp substitution that does some
tweaky "run arbitrary perl code" regexp markup to count the bars.

So then when all was said and done, I had a beautifully organized score in a 
spreadsheet.  Complements of excel, I even used some
nice things, like hiding columns that had stuff that never got changed and took 
up space, dividing the screen left to right so the
instrument names, and their clefs and keys could always be seen adjacent to 
their parts, and a macro for saving both to .xls native
and .txt export at the same time.

I'll attach that .xls (Excel 97) file.  These were the perl5 regexps I used, in 
the form of a DOS batch file:

copy /y %1.txt %1.ly
@
@REM revert bizzare excel quoting behavior:  (fails on real "")
@REM perl -S snr.pl -m %1.ly address@hidden"("\\context".*?\>)\"@address@hidden
perl -S snr.pl -m %1.ly s@(?<=\t)\"([^\t]+?)\"(?=[\t\n])@address@hidden
perl -S snr.pl -m %1.ly address@hidden"\"@\"@g
@
@REM pattern match for cells that look like measures and
@REM insert barchecks between them:
perl -S snr.pl -m %1.ly 
s@(?<!transpose)(?<!key)\t([a-grs]([ei]s)?[,']*[0-9]*)(?=\t)@\t$1 
address@hidden
@
@REM substitute uninterrupted multiple bars rest
perl -S snr.pl -m %1.ly 
s@(?{$cnt=1;})(RR([0-9\.]*)([\s&pipe;]+RR([0-9\.]*)(?{$cnt=$cnt+1;}))*)@R$2\*$cnt
 @g


snr.pl is a script that I had already wrote to simply do a regexp substitution 
on a whole file, either catting to the screen or
mutating the file if -m is specified.  I refuse to use sed's different and less 
powerful regexp system, and -m is included because
command.com is too braindead to pipe stuff in complicated ways.  This is almost 
always good enough.  Also, the < symbols are
real, not an email artifact.  I wrote them into snr.pl because command.com is 
also so braindead that all <, | and > symbols have
piping behavior and cannot be quoted.  I will attach snr.pl also.

BTW- please do not flame me for not using python.  I already had written this 
snr.pl thing which works perfect for the task.  I
also don't know python and don't really care, even though I hate perl too and 
understand why you might.

Notice that the rest symbol that I used was RR1 or RR2. etc instead of R1 or 
R2. since that didn't break the ability to do the old
thing.

Okay, that's all for now.  Take a look.  Ask me if you have any questions of 
how I made it work, or if you have suggestions for how
I might make my life and others easier.  Oh, and if there are people with whom 
my comments about GUIs really resonate and anybody
wants to go after a modern (up to par with MS Word or better), progressive (>5x 
human efficiency increase over finale et al), music
notation GUI that can export to lilypond, send me an email.  I have a lot of 
ideas tossing around, but nothing of useful general
interest yet.

Oh, by the way, if you compile the arrangement remember that it is missing a 
rhythm section and lily's midi is missing swing!
Quantized midi is such a horrible representation of this sort of music!


Jeff Henrikson

tenderly.xls
Description: MS-Excel spreadsheet

snr.pl
Description: Binary data