bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 10.08.2022 20:19, Eli Zaretskii wrote:
> > Date: Wed, 10 Aug 2022 20:02:20 +0300
> > Cc: 56682@debbugs.gnu.org, gregory@heytings.org, stephen.berman@gmx.net,
> >   monnier@iro.umontreal.ca
> > From: Dmitry Gutov <dgutov@yandex.ru>
> >
> > It's much worse. In the aforementioned dictionary.json, with
> > fully-featured font-lock (narrowing disabled in handle_fontified_props)
> > the buffer is perfectly responsive to edit as long as I (setq
> > bidi-inhibit-bpa nil). Even with show-paren-mode enabled.
> >
> > Without that, however, simple navigation commands take a significant
> > amount of time. These measurements are with show-paren-mode off:
> >
> > Near BOB:
> >
> > (benchmark 1 '(next-line 1)) =>
> > Elapsed time: 0.246551s
> > Elapsed time: 0.247237s
> > Elapsed time: 0.247392s
> >
> > Near EOB:
> >
> > (benchmark 1 '(next-line 1)) =>
> > Elapsed time: 0.059583s
> > Elapsed time: 0.040372s
> > Elapsed time: 0.059508s
>
> That's what I meant: it isn't a catastrophe.  1/4th of a second for
> C-n is barely perceptible.

It's 2.5 the upper limit on what's perceived as "instant". And it's 
higher than the "uncomfortable" font-lock delays I have seen.

> "Catastrophe" in my book is when it takes more than a couple of
> seconds, like 10 sec or more.

For such a basic operation as next-line, it seems excessive to me. But 
if you're sure about this, all right.

This delay doesn't seem to extend to other commands, such as C-f/C-b or 
self-insert-command. It does seem to affect isearch, though.

> > > not
> > > like we've seen with the files we used for the long-lines speedups.
> > > And if someone does think it's a catastrophe, they can always disable
> > > the BPA, perhaps even globally,
> >
> > That reminds me of "the user can always disable font-lock" and your
> > dismissal of Alan's advice to change font-lock-maximum-decoration to 2,
> > saying that we should have good editing performance OOTB.
>
> No, because font-lock is much more important than the effect of the
> BPA.  (Do you even understand what the BPA does, and did you ever see
> it in action?  Without that, this part of the discussion is just waste
> of time.)

I have looked at examples now. Seems important for text in pertinent 
human languages.

> > > if they know they will never need to
> > > look closely at bidirectional text: the effects are hardly visible
> > > unless you actually read the text.
> >
> > We could add (setq bidi-inhibit-bpa nil) to prog-mode, I suppose.
>
> Not to prog-mode, that I won't agree.  But maybe for JSON files, if we
> think they are unlikely to suffer.

Sure, maybe.

Where do we draw the line, though? Not in prog-mode because a .js file, 
say, can have comments in Persian or Hebrew?

By that measurement, JSON is fine (no comments supported). But JSON 
contains a bunch of strings, could store localizations, etc.
I suppose the contents of said strings can store i18n text, with the 
expectation of a user being able to edit it.

> > It already has a bidi-paragraph-direction setting anyway.
>
> That's not even similar.  That's just saving Emacs to figure that out
> itself, wasting CPU cycles on what is known in advance.  Because a PL
> buffer _always_ has its paragraphs left-to-right.
>
> > > This is impossible without a complete redesign of how the bidi display
> > > works: it saves no information in the buffer object, none whatsoever.
> > > The information is recomputed every time the display code is called,
> > > and only for the portion of the buffer that needs to be traversed or
> > > displayed; then it is discarded.  There are no caches of any kind that
> > > keep the information after redisplay has done its job.
> >
> > Perhaps a cache similar to syntax-ppss one could do the job? Not sure
> > what would be the trafeoffs, though. Like, how that would change the
> > performance in the simple case.
>
> There are no caches for a good reason: the Unicode Bidirectional
> Algorithm that we implement results in very far-reaching non-local
> effects of small changes.  Even inserting or deleting a single
> character can dramatically change how the buffer looks on display very
> far from the locus of the change.

In both directions?

> Figuring out which changes
> invalidate which caches is extremely non-trivial, and in many cases
> requires the same amount of work as would take to just recompute
> everything from scratch.

Sounds a little similar to syntax-ppss cache: changing a character near 
BOB invalidates the whole cache for all positions > POS.

This one sounds more complex, though.