[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Serious feedback and improvement headroom

From: Urs Liska
Subject: Serious feedback and improvement headroom
Date: Fri, 04 Apr 2014 12:43:18 +0200
User-agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Thunderbird/24.4.0

Hi devels,

I think I have some serious feedback to share, and it may lead to even more feedback and input.

This week I had the opportunity to present myself, LilyPond and my "plain text workflows" topic to a significant group of the Henle staff. In order not to whet your appetite too much: Currently there isn't a chance to get a foot into that business. They have very strict structures, and practically everything I would have to offer is very incompatible with these. So for them adopting LilyPond would mean throwing mostly everything out that they have built over the past decades. Nevertheless they were very interested and _did_ see the point in using version control for edition projects. Actually they even realized that this would vaporize a number of annoying issues they are facing every day ...

Apart from not having had an "accountable" success I had a paid trip to Munich, a few hours in Munich's sun - and I'm now proud owner of a beautiful engraving plate of m. 26-30 of the second movement of Chopin's f minor concerto :-)


The most interesting aspect of the meeting was that Henle's (only) in-house engraver was present too, and this may become a fruitful contact. He is using Amadeus, a Linux (!) program he bought for 20.000 Euro in 1988 and which has been out of development for something like 15 years now. IIUC they have two external teams of engravers, and those _do_ use Finale and Sibelius.

Amadeus is surprisingly similar to LilyPond in many respects, and as he's aware that he may not continue to use Amadeus forever LilyPond might be the obvious choice to switch to. However he had a number of questions and aspects where he considers Amadeus (still) significantly superior, and we should do good in trying to get as much input out of that contact as possible. I think _any_ comment from an engraver who's making scores for Henle day by day and uses a very similar tool to ours should be taken very seriously.


I got the Amadeus manual and will probably be able to share it with you, but I didn't get an answer so far.

First: Amadeus is a compiled system with an input language that is in parts extremely similar to LilyPond's while in others absolutely not. Other than with LilyPond input is by design edited in different files that are processed with different editors and tools. I'd say you have separate files/editors for things like \header {} or \paper {} etc. and for the actual music. You edit these files with editors that (from the manual) resemble editors like emacs or vim - with lots of control commands. Particularly interesting is that (IISC) you write all layout tweaks in a separate file - exactly what we're working on with Jan-Peter's edition-engraver! For viewing the results a shell script processes the files by preprocessing the parts, merging them to a preliminary format and finally producing an eps or pdf file. This looks like a traditional Unix pipe chain..

The music description itself is extremely similar to LilyPond, with a number of characteristic differences. I think that some of these are mainly a matter of taste or habit, some of them are (IMO) better in LilyPond (the handling of accidentals in the input for example: In Amadeus you write the "visible" notes, that is: the meaning of "d f a" depends on the currently active key signature). But some others look interesting, and I'll post them on bug-lilypond to discuss if some of them might warrant a feature request. But beyond the basic music items there are significant conceptual differences in the input language. In general LilyPond's language is more verbose than Amadeus'. A few examples:

treble F 4/4 is the equivalent to

\clef treble
\key f \major
\time 4/4

(,9 .... ),10_a140140h30
seems to be an equivalent to a \shape invocation

Personally I think that LilyPond's approach is a very good compromise between Amadeus' assembler-like appearance and the confusing verbosity of e.g. XML formats. But this is something Henle's engraver would consider a significant problem because he thinks that he needs considerably less time entering the music in Amadeus.

This becomes particularly important when it comes to tweaking output. He wrote (my translation): "My first look at LilyPond [through my presentation and a follow-up email] shows similarities to Amadeus, but OTOH I have the suspicion that the operation isn't sufficiently mature to allow economic ans fast work. I think, you just have to input too much information to get an optimal result." But he's also arguing at the level of actually entered number of characters. As a response to my argument that I prefer explicit pitches (a fis is a fis, regardless of needing a sharp or not, while in Amadeus a pitch is only defining a notehead position) he sais: "I don't understand why I should always enter the accidentals explicitly when they are already defined by the key. When I'm seeing the current key in the score display, I do know that in es major the a is actually an as, isn't it? Apart from that the editor always shows me the currently active key. The encoding in Amadeus is always unambiguous and can't produce misunderstandings. I have the impression that LilyPond always needs twice or three times the characters for the same content. That's offputting ..." While I still think that explicit pitches are better for a number of reasons this _is_ the way someone thinks who has to produce 1.500-2.000 pages of high-quality scores per year. For him each additional character makes a difference.

I don't suggest any significant changes in our input syntax. But I want to point out that editing efficiency on that level _is_ an issue we should keep taking into account when it comes to professional work. For this guy it makes a difference if he can (thousands of times) type "ho" instead of "\stemUp". And we all know that the process of tweaking output isn't that straightforward with LilyPond (although I very much appreciate all the little and bigger improvements we constantly see).


Another aspect that I found very astonishing is compilation speed. Of course Amadeus files have to be compiled before the result is visible, and this can be automatically done upon save. I think Frescobaldi's behaviour with Ctrl-M and by now the autocompile is an equivalent approach here. But the engraver claims that recompilation of a 30 page scores needs 2/10 of a second on an average computer. So this _is_ a fundamental difference, because he _does_ have practically instant WYSIWYG while still benefitting from text input and the compiled approach. I don't have a clue as to how Amadeus achieves this. What comes to mind at first is a very smart approach as to what has to be recalculated and what not. As Amadeus also creates high-quality engraving I can't really imagine that this difference is only due to efficient programming. Although I can imagine that a program developed in the 1980s has a significantly stricter approach to runtime efficiency. But I can also imagine that it has to do with the way C++ and Scheme code interact with LilyPond. In another context I see a similar thing with LaTeX: Compiling a file with lualatex and fontspec takes longer by orders of magnitude than with plain latex. So maybe we really have a conceptual issue with the efficiency of LilyPond's runtime work.

As I can't imagine that we can speed up LilyPond's processing time by 95% I really think we should find ways for partial recompilation. This is reportedly also the approach the new Steinberg application takes. For example when entering music I could do with only updating the currently interesting section (maybe a system would be a good unit for that). Maybe it would be possible to create compilation modes that have a significantly sloppier approach to breaking. There are many stages in the development of a score where I don't care at all about the layout of the complete score. I recall that in Finale you could (probably: can) switch between score and roll mode (however this has been called). The latter was just one system without any breaks, and that is what can be used for music entry and for all kinds of _content_ editing. Of course this takes significantly less amounts of processing power, and it would be _much_ easier to create a way to only compile the currently changed measure (or maybe some more if there are living spanners).

Using strategies with skipTypesetting can help a lot in saving compilation time. But it would be an extremely useful enhancement if we could get some kind of partial recompilation _within_ the context of an existing score. [I just got another email indicating that Amadeus will by default recalculate the current page only, and this works in near instant time. I think with this we can get to an area where calculating efficiency can make such a difference.] What I would vote for very much is an option to recalculate the current system only, but with outputting the complete score anyway (however it may be determined what "current" is). This should work in many cases when a modification doesn't change line breaking. And it would dramatically increase the editing experience.


This email contains a lot of stuff, and I expect it to raise considerable discussion. Therefore I suggest to try keeping it organized and updating message titles as soon as we're discussing individual topics. I also suggest to respond to different topics separately, with creating recognizable subthreads. Otherwise it may soon become impossible to find thoughts you recall having read about ;-)


reply via email to

[Prev in Thread] Current Thread [Next in Thread]