[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Serious feedback and improvement headroom

From: Urs Liska
Subject: Re: Serious feedback and improvement headroom
Date: Fri, 04 Apr 2014 16:18:21 +0200
User-agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Thunderbird/24.0

*Von:* David Kastrup <address@hidden>
*Gesendet:* Fri Apr 04 14:24:28 MESZ 2014
*An:* Urs Liska <address@hidden>
*CC:* LilyPond Development Team <address@hidden>, "Jan Warchoł"
*Betreff:* [SPAM] Re: Serious feedback and improvement headroom

Urs Liska <address@hidden> writes:

    The most interesting aspect of the meeting was that Henle's (only)
    in-house engraver was present too, and this may become a fruitful
    contact. He is using Amadeus, a Linux (!) program he bought for 20.000
    Euro in 1988

Well, in 1988 there was no Euro.  I presume that you have just converted
by some factor.

Of course. He said 40.000 Mark.

But in 1988 there also was no Linux.  Obviously,
conversion by a factor would not help with that.

Well, I don't know anything about the history of that program, and it's hard to find much on the net.
I know Amadeus does run on Linux, and presumably it was developed on Unix.

    But beyond the basic music items there are significant conceptual
    differences in the input language. In general LilyPond's language is
    more verbose than Amadeus'. A few examples:

    treble F 4/4 is the equivalent to

    \clef treble
    \key f \major
    \time 4/4

    (,9 .... ),10_a140140h30
    seems to be an equivalent to a \shape invocation

    Personally I think that LilyPond's approach is a very good compromise
    between Amadeus' assembler-like appearance and the confusing verbosity
    of e.g. XML formats. But this is something Henle's engraver would
    consider a significant problem because he thinks that he needs
    considerably less time entering the music in Amadeus.

    This becomes particularly important when it comes to tweaking
    output. He wrote (my translation): "My first look at LilyPond [through
    my presentation and a follow-up email] shows similarities to Amadeus,
    but OTOH I have the suspicion that the operation isn't sufficiently
    mature to allow economic ans fast work. I think, you just have to
    input too much information to get an optimal result." But he's also
    arguing at the level of actually entered number of characters.

That's something better addressed by input tools rather than the
language.  Emacs would be a good building block if you are used to
UNIX-like systems (and if you ported some large application for generic
UNIX to GNU/Linux, you probably have a bit of exposure here).  It's been
a long time since I looked at what lyqi, the "LilyPond quick input" mode
by, uh, Nicolas Sceaux?, did.

Partly yes. When it comes to creating the input text for tweaks I agree that editing tools can do a lot to improve things. But when you have the strong opinion that typing "f" is an improvement over typing "fis" there's not much editing tools can do for you.

    As a response to my argument that I prefer explicit pitches (a fis is
    a fis, regardless of needing a sharp or not, while in Amadeus a pitch
    is only defining a notehead position) he sais: "I don't understand why
    I should always enter the accidentals explicitly when they are already
    defined by the key. When I'm seeing the current key in the score
    display, I do know that in es major the a is actually an as, isn't it?

Oh, he's perfectly right about that.  If you are writing a _score_, it
is reasonable to write down what you see if your goal is to reflect a
manuscript.  With LilyPond, the focus is on describing music rather than
programming output, however.  There are advantages to that since I am
not fixed in the key signature.  I can easily decide at a late stage of
publishing whether I'll be writing Bach's "Dorische" in the rather
unmotivated dorian key or whether I just punt and print it in d minor.

I can use the same source for a number of different styles including the

That's exactly my opinion. Probably he has never prepared a score for a different purpose than a single printed edition, so the flexibility of differen accidental styles wouldn't matter to him. Anyway, I find the idea of the d major chord <d f a> _very_ strange ...
And that's why I wouldn't want to suggest any changes in that area.

    I have the impression that LilyPond always needs twice or three times
    the characters for the same content. That's offputting ..."

Introduce him to MusiXTeX.

That's not the point. He's (rightly) only interested in the comparison to Amadeus.

    While I still think that explicit pitches are better for a number of
    reasons this _is_ the way someone thinks who has to produce
    1.500-2.000 pages of high-quality scores per year. For him each
    additional character makes a difference.

Input tool problem.

I don't think so.
Take the "ho" example, which is shorter than "\stemUp".
You can define a wrapper and write "\ho", which is still longer than "ho". The only thing to come to par with the two characters would be a keyboard shortcut. But of course it's not a good idea to create shortcuts for all of these commands. That would be way over the top. I think he is right in saying that with LilyPond you need more typing than with Amadeus.

But I think that's not the real point. I think it's a tradeoff between number of characters to type, clarity and verbosity. And I do think that Lily's approach is generally very good here.

It only becomes a text format problem if the
verbosity impacts your _reading_ speed or comprehension.  Which is one
of the reasons many programmers don't like COBOL.  I don't think that
LilyPond is really too verbose, and you can of course write a lot of
shortcuts at the price of making it harder to pass scores around.  But
should be no problem for in-house use.

    I don't suggest any significant changes in our input syntax. But I
    want to point out that editing efficiency on that level _is_ an issue
    we should keep taking into account when it comes to professional

Current LilyPond input tools, editors, and workflows suck.  That is not
LilyPond's "responsibility" as a command line tool, but of course it is
something affecting its uptake nevertheless.

    Another aspect that I found very astonishing is compilation speed. Of
    course Amadeus files have to be compiled before the result is visible,
    and this can be automatically done upon save. I think Frescobaldi's
    behaviour with Ctrl-M and by now the autocompile is an equivalent
    approach here. But the engraver claims that recompilation of a 30 page
    scores needs 2/10 of a second on an average computer. So this _is_ a
    fundamental difference, because he _does_ have practically instant
    WYSIWYG while still benefitting from text input and the compiled
    approach. I don't have a clue as to how Amadeus achieves this.

There is nothing to it, really.  If your account of Amadeus' history is
correct, it originates at a time when the typical UNIX workstation it
would be running on would be a 16MHz 68000 CPU.

This is a professional product used by professionals.  So the core
functionality will have been written in a manner where the professionals
are not reduced to waiting weeks for a proof.  My guess would be that
the order of magnitude would have been something like a minute per page.
With the performance available at that time, we would not have gotten
our character metrics out of pango in that time, let alone get anything

Much of the internals of LilyPond are inefficient, and the glue layers
and data structures of Scheme also come at a cost (and using
not exactly renowned to be an efficient Scheme implementation, also
plays into that).  But Scheme is also giving us building blocks for
solving a lot of stuff without recompilation.

OK, that may well be. But in effect it's a usability issue that shouldn't be underestimated. near-realtime WYSIWIG is an important factor, particularly when you have to tweak things iteratively by trial-and-error. Against Finale users we can always say that the compiled approach has inherent advantages that are worth the waiting time. But in this case that doesn't count.

    What comes to mind at first is a very smart approach as to what has to
    be recalculated and what not. As Amadeus also creates high-quality
    engraving I can't really imagine that this difference is only due to
    efficient programming. Although I can imagine that a program developed
    in the 1980s has a significantly stricter approach to runtime

We are not talking "stricter approach" here.  We are talking
"requirement".  One of the advantages of slow iron is that "I'll worry
about the performance implications of this some other time" is not an
option.  And that's pervasive in LilyPond.

    But I can also imagine that it has to do with the way C++ and Scheme
    code interact with LilyPond.

Both C++ and Scheme take their toll, yes.

    In another context I see a similar thing with LaTeX: Compiling a file
    with lualatex and fontspec takes longer by orders of magnitude than
    with plain latex.

If you want a real eye-opener, run weave tex.web && tex tex.tex and take
a look at the number of seconds taken per page (assuming all fonts are
available).  That's the performance needed in the 80s.  LaTeX is not
even close.  It's of course still damn fast when compared with today's
expectations since it is running on a damn efficient engine, and even if
engine has to through a lot of ridiculous hoops, it gets the job
done faster than current programs.

    So maybe we really have a conceptual issue with the efficiency of
    LilyPond's runtime work.

In the current environment?  I don't think all that much.  The important
thing remains the creative process, and that does not (or should not)
involve running LilyPond.

I don't understand what you mean. You mean the creative process of preparing the content of a score takes more time than compiling the LilyPond file? We're talking about producing an average of ~10 publication quality pages per working day, and for this it does make a difference if you have to wait 0.2 or 12 seconds to get visual feedback for your editing.

We _are_ too slow in our backend.  No question about that.  Parsing a
file is much faster than typesetting it.  And the parsing still could be
sped up considerably regarding the Scheme/LilyPond synchronization.

    As I can't imagine that we can speed up LilyPond's processing time by
    95% I really think we should find ways for partial recompilation. This
    is reportedly also the approach the new Steinberg application
    takes. For example when entering music I could do with only updating
    the currently interesting section (maybe a system would be a good unit
    for that).

LilyPond is a command line application.  Partial recompilation does not
really map sensibly to it.

But there might be steps that can help. I will write more about that in response to Jan-Peters message.

If you write using LaTeX, check out
<URL:>.  That's
technology that had to run on 133MHz CPUs at the height of its
development and shows interesting and actually usable approaches to
editor-supported partial compilation.  It should be a reasonably doable
effort (actually also a nice GSoC-size project) to make preview-latex
available for LilyPond-book.  In a first step, for LaTeX documents, in a
second one, working with LilyPond
documentation in Texinfo.  It would
make editing our docs so much more pleasant that it isn't funny.

But I digress.  Check out its usability for typical math documents and
their editing, and in particular recreating formula displays after

Most particularly: check out the performance when dvipng is installed
and you are _not_ going through PDFLaTeX but rather straight LaTeX and
DVI files.  That's an 80s data flow rather than "modern font handling",
and it's basically instantaneous.

    Maybe it would be possible to create compilation modes that have a
    significantly sloppier approach to breaking. There are many stages in
    the development of a score where I don't care at all about the layout
    of the complete score. I recall that in Finale you could (probably:
    can) switch between score and roll mode (however this has been

You can do that in LilyPond as well.  The resulting output is not
necessarily all that handy for typical previewers.

    Using strategies with skipTypesetting can help a lot in saving
    compilation time. But it would be an extremely useful enhancement if
    we could get some kind of partial recompilation _within_ the context
    of an existing score.

That kind of stuff needs to get integrated into editing tools and modes
since only editing tools and modes have the effortless information about
what is currently on-screen and/or edited.

    [I just got another email indicating that Amadeus will by default
    recalculate the current page only, and this works in near instant
    time. I think with this we can get to an area where calculating
    efficiency can make such a difference.]

Still needs integration with the editing tool.

    What I would vote for very much is an option to recalculate the
    current system only, but with outputting the complete score anyway
    (however it may be determined what "current" is). This should work in
    many cases when a modification doesn't change line breaking. And it
    would dramatically increase the editing experience.

There is little point in investing a lot of effort here when the current
editing tools do not even deal with skipTypesetting and other things.

Well, skipTypesetting is not really what would be needed.
I can manually enter the three commands to let LilyPond compile one system of an existing score. And we surely can talk Frescobaldi to do that automatically. But the result won't be that single system re-engraved but the music of the system treated as the beginning of the score, with titles and indentation, instrumentnames and whatnot.

Frescobaldi is the only LilyPond text editor that is actively developed.

And I would try hard to help with that.
Not because I would want to talk that engraver into using LilyPond, but because that's only a new push toward an issue we have seen for quite some time.

As said I'll write some more in another email.


Denemo could probably make use of some of that
technology, but as it has
its own display engines, there is not a lot of pressure to get partial
updates for it.

David Kastrup

reply via email to

[Prev in Thread] Current Thread [Next in Thread]