Re: Rethinking the design of xwidgets

[Eli Zaretskii]

> From: Akira Kyle <ak@akirakyle.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 13 Oct 2020 14:37:31 -0600
> 
> > I'll need to read up on those two topics, but in general, as 
> > someone
> > already pointed out, the Emacs display of images supports
> > interactivity by letting you call functions triggered by mouse
> > gestures on the varipous spots on the image.  If such a function
> > produces a zoomed-in/out image and displays it instead of the 
> > original
> > one, wouldn't that fulfill the need?  E.g., we already support
> > animated GIF images and zoom-in/out by mouse scroll, so it is
> > inaccurate to say that we can only display static images.
> 
> Can actions be triggered on a click and drag as one would do to 
> pan within a plot?

I didn't have time to try that, but just looking at the code, I don't
see anything that makes the clocks on images special, so I'd expect
the same modifiers and drag-detection code to work.  And if it
doesn't, I seed no reason why we couldn't teach it to do so.

> As far as I know there are really only two FLOSS PDF libraries out 
> there: poppler and muPDF. Between them poppler supports more PDF 
> features and seems to be more widely used. I agree this isn't 
> really as compelling of a use case for xwidgets, but I seems like 
> it would be easier to achieve better PDF rendering via xwidgets.

I can see how that is tempting: the image handling is taken care of.
What I'm saying is that this advantage should be carefully weighed
against the disadvantages and difficulties in integrating such
self-managing objects into the Emacs display code.

> > E.g., the kludge in dispnew.c around line 4365.  It disables one
> > of the most important redisplay optimizations in Emacs, once you
> > build with xwidgets enabled.
> 
> Ah, I had briefly looked at that but I wasn't sure how important 
> it was. I may be able to make some progress on that but it's hard 
> to understand the context for that function. It seems like that 
> function is responsible for an optimization that avoids redrawing 
> glyphs if the window is being scrolled by copying them

Not only if the window is scrolled, but also if the previous display
can be converted into the current display by scrolling some of the
stuff that is already on the glass.  the algorithm is a basic
comparison of the current and desired display, similar to what the
Diff utility does.

> however I couldn't find where or how the glyphs are actually copied?

See the calls to rif->scroll_run_hook.

> > I'm not trying to scare you, mind you, just point out that the
> > ideas you have should probably be talked through in mode detail,
> > to see that they are workable.  For example, did you study the
> > various window-redisplay optimizations, and if so, did you think
> > how the widgets will adapt to them?
> 
> I'm just starting to dig my teeth into some of that. However I'm 
> wondering if anyone has more recently tried to quantify the 
> improvements the various redisplay optimizations make to both 
> actual and perceived render time of Emacs. For example the comment 
> on the scrolling_window function in dispnew.c that you pointed me 
> too references that it is implementing an algorithm from 1978, 
> perhaps it would be worth exploring if there are any algorithmic 
> advances?

I don't think that algorithm is less optimal nowadays than it was back
then; the really interesting question is how much it saves us when
compared to simply redisplaying all those lines.

> It strikes me that perhaps some optimizations may no longer be as
> necessary on modern hardware or on GUI displays as they were when
> running emacs in a VT on 20 year hardware.

Those are very good questions.  I'm not aware of any such
investigations since the current redisplay optimizations were
implemented around 20 years ago: at that time, Gerd Möllmann, who
developed the current display engine, did add optimizations one by one
until he got reasonable redisplay performance.

Measuring the speedups from each of the several optimizations is an
important job, but it's a large job.  For starters, one needs to come
up with a large enough and representative enough sample of redisplay
use cases, and that is not easy.  So I'd encourage you (or anyone
else, actually) to do this important job, but be aware that it could
take non-trivial time and effort.

> Conversely there may be further optimization that could be done by
> taking advantage of more recent hardware advances such as offloading
> to a GPU or utilizing vectorized instructions.

We try not to use machine-dependent code in Emacs, because that's a
maintenance burden, what with today's fast pace of chip development
and obsolescence.  Vectorization is generally left to optimizing
compilers, and relying on special hardware, such as GPU, is not
something we should depend on directly.  We should instead hope that
the GUI toolkits and display systems we use will do that for us.

> If there's one think I've taken away from my CS education, its that
> indeed "premature optimization is the root of all evil".

The optimizations we have today were definitely NOT premature when
they were introduced.  How much they are still needed today is indeed
an interesting and important question that still awaits the motivated
investigators.  I have only indirect evidence that some of the
optimizations still do a useful job: compare the redisplay performance
when linum-mode is turned on with the performance when the native
display-line-numbers-mode is used instead.  Also, we frequently hear
complaints about redisplay performance, even if you take away the
problems with long lines.