bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 02.08.2022 15:35, Eli Zaretskii wrote:

> > - syntax-ppss is also important for correctness: for commands to
> > understand whether the point is inside a string, comments, etc. So it's
> > better to avoid applying narrowing when calling it. Unless you're in a
> > multiple-major-modes situation.
> > - font-lock calls syntax-ppss.
>
> I believe I was talking about syntax-ppss being called from font-lock,
> indeed.  Before Gregory's changes, if you visit a large file with very
> long lines, and interrupt Emacs while it is non-responsive, you will
> in many/most cases find yourself in syntax-propertize or its
> subroutines, and you will see that it is almost always called to
> traverse the entire long line.

Interrupt it right after pressing 'M->'? Or at any time during editing 
the buffer later?

The latter really shouldn't happen. If it does, perhaps it's the result 
of narrowing during redisplay, which might blow syntax-ppss's caches.

In any case, if you could point to a scenario and the revision to test 
it on, I'll be sure to take a look.

> > So ideally font-lock is either called with undo-able narrowing, or is
> > simply passed a range of positions, and shouldn't fontify too far from them.
>
> Many major-modes do widen the buffer, though.

Whether they do or not, font-lock widens by default, see 
font-lock-dont-widen.

> > The latter seems to be the case already (if you open xdisp.c and press
> > M->, only top and bottom of the buffer are fontified)
>
> It is not enough to look for faces in order to realize how much of the
> buffer was scanned.

I evaluated (next-single-property-change 1 'fontified), when near BOB 
and when near EOB.

> > with the caveat that font-lock always tries to backtrack to BOL when
> > fontifying the current hunk. Which makes sense, of course, but could
> > be tweaked for long lines to avoid re-fontifying the whole buffer
> > again and again.
>
> "Tweaked" how?

15b2138719b34083 is one example.

> > IOW, IIUC the fix for font-lock performance could be better implemented
> > inside font-lock itself, as long as all the info about whether the
> > current line is "long" is available to Lisp.
>
> No one will object to making font-lock faster.  But the experts who
> can do that are few and far in-between, and seem to have other itches
> to scratch, since these issues are known for a long time, and several
> times were even discussed at length.

The fact that we have +1 contributor to the C part of Emacs (the display 
engine, etc), and a successful one at that, does nothing about the fact 
that Lisp is easier to write and debug.

If we're able to demonstrate that the remaining bottlenecks are inside 
font-lock, it should be easier to improve there.