[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Serious feedback and improvement headroom
Re: Serious feedback and improvement headroom
Fri, 04 Apr 2014 16:18:21 +0200
Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Thunderbird/24.0
*Von:* David Kastrup <address@hidden>
*Gesendet:* Fri Apr 04 14:24:28 MESZ 2014
*An:* Urs Liska <address@hidden>
*CC:* LilyPond Development Team <address@hidden>, "Jan Warchoł"
*Betreff:* [SPAM] Re: Serious feedback and improvement headroom
Urs Liska <address@hidden> writes:
The most interesting aspect of the meeting was that Henle's (only)
in-house engraver was present too, and this may become a fruitful
contact. He is using Amadeus, a Linux (!) program he bought for 20.000
Euro in 1988
Well, in 1988 there was no Euro. I presume that you have just converted
by some factor.
Of course. He said 40.000 Mark.
But in 1988 there also was no Linux. Obviously,
conversion by a factor would not help with that.
Well, I don't know anything about the history of that program, and it's
hard to find much on the net.
I know Amadeus does run on Linux, and presumably it was developed on Unix.
But beyond the basic music items there are significant conceptual
differences in the input language. In general LilyPond's language is
more verbose than Amadeus'. A few examples:
treble F 4/4 is the equivalent to
\key f \major
(,9 .... ),10_a140140h30
seems to be an equivalent to a \shape invocation
Personally I think that LilyPond's approach is a very good compromise
between Amadeus' assembler-like appearance and the confusing verbosity
of e.g. XML formats. But this is something Henle's engraver would
consider a significant problem because he thinks that he needs
considerably less time entering the music in Amadeus.
This becomes particularly important when it comes to tweaking
output. He wrote (my translation): "My first look at LilyPond [through
my presentation and a follow-up email] shows similarities to Amadeus,
but OTOH I have the suspicion that the operation isn't sufficiently
mature to allow economic ans fast work. I think, you just have to
input too much information to get an optimal result." But he's also
arguing at the level of actually entered number of characters.
That's something better addressed by input tools rather than the
language. Emacs would be a good building block if you are used to
UNIX-like systems (and if you ported some large application for generic
UNIX to GNU/Linux, you probably have a bit of exposure here). It's been
a long time since I looked at what lyqi, the "LilyPond quick input" mode
by, uh, Nicolas Sceaux?, did.
Partly yes. When it comes to creating the input text for tweaks I agree
that editing tools can do a lot to improve things. But when you have the
strong opinion that typing "f" is an improvement over typing "fis"
there's not much editing tools can do for you.
As a response to my argument that I prefer explicit pitches (a fis is
a fis, regardless of needing a sharp or not, while in Amadeus a pitch
is only defining a notehead position) he sais: "I don't understand why
I should always enter the accidentals explicitly when they are already
defined by the key. When I'm seeing the current key in the score
display, I do know that in es major the a is actually an as, isn't it?
Oh, he's perfectly right about that. If you are writing a _score_, it
is reasonable to write down what you see if your goal is to reflect a
manuscript. With LilyPond, the focus is on describing music rather than
programming output, however. There are advantages to that since I am
not fixed in the key signature. I can easily decide at a late stage of
publishing whether I'll be writing Bach's "Dorische" in the rather
unmotivated dorian key or whether I just punt and print it in d minor.
I can use the same source for a number of different styles including the
That's exactly my opinion. Probably he has never prepared a score for a
different purpose than a single printed edition, so the flexibility of
differen accidental styles wouldn't matter to him. Anyway, I find the
idea of the d major chord <d f a> _very_ strange ...
And that's why I wouldn't want to suggest any changes in that area.
I have the impression that LilyPond always needs twice or three times
the characters for the same content. That's offputting ..."
Introduce him to MusiXTeX.
That's not the point. He's (rightly) only interested in the comparison
While I still think that explicit pitches are better for a number of
reasons this _is_ the way someone thinks who has to produce
1.500-2.000 pages of high-quality scores per year. For him each
additional character makes a difference.
Input tool problem.
I don't think so.
Take the "ho" example, which is shorter than "\stemUp".
You can define a wrapper and write "\ho", which is still longer than
"ho". The only thing to come to par with the two characters would be a
keyboard shortcut. But of course it's not a good idea to create
shortcuts for all of these commands. That would be way over the top.
I think he is right in saying that with LilyPond you need more typing
than with Amadeus.
But I think that's not the real point. I think it's a tradeoff between
number of characters to type, clarity and verbosity. And I do think that
Lily's approach is generally very good here.
It only becomes a text format problem if the
verbosity impacts your _reading_ speed or comprehension. Which is one
of the reasons many programmers don't like COBOL. I don't think that
LilyPond is really too verbose, and you can of course write a lot of
shortcuts at the price of making it harder to pass scores around. But
should be no problem for in-house use.
I don't suggest any significant changes in our input syntax. But I
want to point out that editing efficiency on that level _is_ an issue
we should keep taking into account when it comes to professional
Current LilyPond input tools, editors, and workflows suck. That is not
LilyPond's "responsibility" as a command line tool, but of course it is
something affecting its uptake nevertheless.
Another aspect that I found very astonishing is compilation speed. Of
course Amadeus files have to be compiled before the result is visible,
and this can be automatically done upon save. I think Frescobaldi's
behaviour with Ctrl-M and by now the autocompile is an equivalent
approach here. But the engraver claims that recompilation of a 30 page
scores needs 2/10 of a second on an average computer. So this _is_ a
fundamental difference, because he _does_ have practically instant
WYSIWYG while still benefitting from text input and the compiled
approach. I don't have a clue as to how Amadeus achieves this.
There is nothing to it, really. If your account of Amadeus' history is
correct, it originates at a time when the typical UNIX workstation it
would be running on would be a 16MHz 68000 CPU.
This is a professional product used by professionals. So the core
functionality will have been written in a manner where the professionals
are not reduced to waiting weeks for a proof. My guess would be that
the order of magnitude would have been something like a minute per page.
With the performance available at that time, we would not have gotten
our character metrics out of pango in that time, let alone get anything
Much of the internals of LilyPond are inefficient, and the glue layers
and data structures of Scheme also come at a cost (and using
not exactly renowned to be an efficient Scheme implementation, also
plays into that). But Scheme is also giving us building blocks for
solving a lot of stuff without recompilation.
OK, that may well be. But in effect it's a usability issue that
shouldn't be underestimated. near-realtime WYSIWIG is an important
factor, particularly when you have to tweak things iteratively by
trial-and-error. Against Finale users we can always say that the
compiled approach has inherent advantages that are worth the waiting
time. But in this case that doesn't count.
What comes to mind at first is a very smart approach as to what has to
be recalculated and what not. As Amadeus also creates high-quality
engraving I can't really imagine that this difference is only due to
efficient programming. Although I can imagine that a program developed
in the 1980s has a significantly stricter approach to runtime
We are not talking "stricter approach" here. We are talking
"requirement". One of the advantages of slow iron is that "I'll worry
about the performance implications of this some other time" is not an
option. And that's pervasive in LilyPond.
But I can also imagine that it has to do with the way C++ and Scheme
code interact with LilyPond.
Both C++ and Scheme take their toll, yes.
In another context I see a similar thing with LaTeX: Compiling a file
with lualatex and fontspec takes longer by orders of magnitude than
with plain latex.
If you want a real eye-opener, run weave tex.web && tex tex.tex and take
a look at the number of seconds taken per page (assuming all fonts are
available). That's the performance needed in the 80s. LaTeX is not
even close. It's of course still damn fast when compared with today's
expectations since it is running on a damn efficient engine, and even if
engine has to through a lot of ridiculous hoops, it gets the job
done faster than current programs.
So maybe we really have a conceptual issue with the efficiency of
LilyPond's runtime work.
In the current environment? I don't think all that much. The important
thing remains the creative process, and that does not (or should not)
involve running LilyPond.
I don't understand what you mean. You mean the creative process of
preparing the content of a score takes more time than compiling the
We're talking about producing an average of ~10 publication quality
pages per working day, and for this it does make a difference if you
have to wait 0.2 or 12 seconds to get visual feedback for your editing.
We _are_ too slow in our backend. No question about that. Parsing a
file is much faster than typesetting it. And the parsing still could be
sped up considerably regarding the Scheme/LilyPond synchronization.
As I can't imagine that we can speed up LilyPond's processing time by
95% I really think we should find ways for partial recompilation. This
is reportedly also the approach the new Steinberg application
takes. For example when entering music I could do with only updating
the currently interesting section (maybe a system would be a good unit
LilyPond is a command line application. Partial recompilation does not
really map sensibly to it.
But there might be steps that can help. I will write more about that in
response to Jan-Peters message.
If you write using LaTeX, check out
technology that had to run on 133MHz CPUs at the height of its
development and shows interesting and actually usable approaches to
editor-supported partial compilation. It should be a reasonably doable
effort (actually also a nice GSoC-size project) to make preview-latex
available for LilyPond-book. In a first step, for LaTeX documents, in a
second one, working with LilyPond
documentation in Texinfo. It would
make editing our docs so much more pleasant that it isn't funny.
But I digress. Check out its usability for typical math documents and
their editing, and in particular recreating formula displays after
Most particularly: check out the performance when dvipng is installed
and you are _not_ going through PDFLaTeX but rather straight LaTeX and
DVI files. That's an 80s data flow rather than "modern font handling",
and it's basically instantaneous.
Maybe it would be possible to create compilation modes that have a
significantly sloppier approach to breaking. There are many stages in
the development of a score where I don't care at all about the layout
of the complete score. I recall that in Finale you could (probably:
can) switch between score and roll mode (however this has been
You can do that in LilyPond as well. The resulting output is not
necessarily all that handy for typical previewers.
Using strategies with skipTypesetting can help a lot in saving
compilation time. But it would be an extremely useful enhancement if
we could get some kind of partial recompilation _within_ the context
of an existing score.
That kind of stuff needs to get integrated into editing tools and modes
since only editing tools and modes have the effortless information about
what is currently on-screen and/or edited.
[I just got another email indicating that Amadeus will by default
recalculate the current page only, and this works in near instant
time. I think with this we can get to an area where calculating
efficiency can make such a difference.]
Still needs integration with the editing tool.
What I would vote for very much is an option to recalculate the
current system only, but with outputting the complete score anyway
(however it may be determined what "current" is). This should work in
many cases when a modification doesn't change line breaking. And it
would dramatically increase the editing experience.
There is little point in investing a lot of effort here when the current
editing tools do not even deal with skipTypesetting and other things.
Well, skipTypesetting is not really what would be needed.
I can manually enter the three commands to let LilyPond compile one
system of an existing score. And we surely can talk Frescobaldi to do
that automatically. But the result won't be that single system
re-engraved but the music of the system treated as the beginning of the
score, with titles and indentation, instrumentnames and whatnot.
Frescobaldi is the only LilyPond text editor that is actively developed.
And I would try hard to help with that.
Not because I would want to talk that engraver into using LilyPond, but
because that's only a new push toward an issue we have seen for quite
As said I'll write some more in another email.
Denemo could probably make use of some of that
technology, but as it has
its own display engines, there is not a lot of pressure to get partial
updates for it.
Message not available
Re: Serious feedback and improvement headroom, Henning Hraban Ramm, 2014/04/04
Re: Serious feedback and improvement headroom, Francisco Vila, 2014/04/05
Re: Serious feedback and improvement headroom, Jan Warchoł, 2014/04/09