Re: Cairo plans

[Han-Wen Nienhuys]

On Mon, Aug 30, 2021 at 6:31 PM Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:
>
> Am Montag, dem 30.08.2021 um 10:16 +0200 schrieb Han-Wen Nienhuys:
> > With MR 897, I have been able to run the doc build through
> > Cairo. Results are very encouraging: the build is faster, while the
> > resulting PDF file is smaller.
>
> As Werner remarked, only due to not using extractpdfmark.

I'll look into PDF processing software. AFAICT, the subsetting retains
the character mapping of the original font, so I think it should be
possible to rewrite it to embed the Emmentaler font once, point all
font references to the full font, and elide all the subsetted
versions.

> Giving timing for a single HTML file is a bit dubious because it
> requires processing all .tely files for cross-references. For the
> influence of Cairo, you really want to compare the time it takes to run
> lilypond-book to get a single .texi file. However, I'd like to remark
> that generating zillions of tiny snippets is not really the kind of
> things users tend to do...

It is the relevant metric here, because the MR enables lilypond-book
processing. It is also relevant for development because our
regtests/CI pipelinek are based on lilypond-book.

We discussed the performance impact of Cairo earlier, but here are
timings for the MSDM score, CPU pinned at 2Ghz.

-dbackend=ps

--ps
real 0m33.043s

--pdf
real 0m34.241s

-dbackend=cairo

--ps
real 0m30.772s

--pdf
real 0m31.245s

-dbackend=svg
real 2m47.836s

> > Open question is how to position Cairo output and what defaults we
> > should provide.
> >
> > * SVG.
> >
> >   The current SVG backend is glacially slow
>
> IIRC the reason is the widespread use of regexes for matching glyph
> nodes. I think this could be done better, and comparing speed isn't
> really fair until then.

I think it is a fair comparison, given that it's comparing a
documented and shipping feature against an experimental feature. It
would be unfair to dismiss the experimental feature due to bugs, but
that's not what's happening here.

The SVG backend works roughly like the PS backend (run Scheme code
that translates stencils to strings, which are dumped one by one to an
output file), so I am willing to bet serious money that you can't get
to perform faster than the PS backend, and by extension, the Cairo
backend. From a quick look, the regexp stuff could be mitigated by
doing proper XML parsing, but then you'd have to add libxml bindings
for GUILE to the build.

However, my main point is that the SVG output is extremely slow, and
in a poor state of maintenance.  The missing feature (relative to
Cairo) is SVG WOFF support, but:

[hanwen@localhost lilypond]$ git describe HEAD
release/2.20.0-1-18-g2a3a1ed05c
[hanwen@localhost lilypond]$ time lilypond   -dbackend=svg -dsvg-woff
input/regression/les-nereides.ly
GNU LilyPond 2.20.0
...
Layout output to
`les-nereides.svg'.../home/hanwen/vc/lilypond/out/share/lilypond/current/scm/output-svg.scm:455:16:
While evaluating arguments to ly:paper-get-font in expression
(ly:paper-get-font paper (quasiquote #)):
/home/hanwen/vc/lilypond/out/share/lilypond/current/scm/output-svg.scm:455:16:
Unbound variable: paper

After looking through our bugs, it turns out this has been broken for
at least 5 years now, (see
https://gitlab.com/lilypond/lilypond/-/issues/4648)

I see no reason to keep the SVG backend alive given that Cairo
achieves the same functionality faster and with less code.

> > Here are my questions:
> >
> > * when could/would we drop the SVG backend?
> >
> > * when could/would we switch the default PS/PDF/PNG backend to cairo?
>
> From my current understanding of missing features, the amount of
> testing the backend can have received (or rather cannot, due to
> novelty), and the nature of bugs that are found (both in Cairo itself
> and the integration in LilyPond), I don't think the backend is
> currently in a state to be used by default. I would highly prefer to
> not mix switching the default backend with switching to Guile 2.2 that
> will already be disruptive enough (yes, it's going slower than I had
> hoped...).

I see your point, but consider the following: the PS backend
represents 3500 fiddlesome Postscript and Scheme code, which is only
exercised by us.

With the cairo solution, the complexity of rendering and font handling
is moved to Cairo, which is much more widely used and better tested.
We are only left with cairo.cc which is much more straightforward. It
should be expected that the cairo solution takes comparatively little
effort to stabilize.

> > * when could/would we drop the PS backend altogether?
>
> I would say that this step requires going to LilyPond 3.0, along with
> removing all the features and commands that cannot be implemented in
> the Cairo backend, or that we don't want to.

We can discuss this in detail later, but I think a major version bump
is not warranted, as we're leaving the music input intact. For
context, the 1.8 -> 2.0 transition happened when changed

  [(c4 c4)]

to

  c4[( c4])

which invalidated most .ly files, requiring manual vetting of the conversion.

> I would propose the following: For the next stable release (whenever
> that will be), it would be good to have the Cairo backend mature enough
> that it can be shipped as a "preview" in the official builds (that will

my feeling is that this state is actually pretty close now already.



-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen