bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 02.08.2022 17:57, Gregory Heytings wrote:
> > Regarding the long-standing bug reports, we did solve a bunch of 
> > issues already. One major one, IIUC, was redisplay of already 
> > fontified text on long lines.
>
> Try to open the dictionary.json with Emacs on master a month ago.  It's 
> a small file (only 18 MB).  On my computer just opening the file with 
> emacs -Q takes 220 seconds.  220 seconds during which Emacs is 
> completely locked, because of font-lock mode.  If you're not convinced, 
> turn font-lock mode off, open the file, and turn font-lock mode on.

I downloaded that file, and I commented out the code in 
'handle_fontified_prop' which performs the narrowing on master.

And recompiled, leaving all the other settings the same.

Visiting dictionary.json takes about 1 second.

M-> takes ~2 seconds, but only the first time (and until the next 
modification near the beginning of the buffer, I guess).

Scrolling is as fast as without my change. All fontification seems 
correct (which is not the case on master).

> > Another piece of the puzzle was added by Stefan in 15b2138719b340.
>
> That looked promising, but sadly it had only a very limited effect.
>
> > So perhaps we should re-evaluate the testing scenario to see where the 
> > current bottlenecks are. If we current main issue is the 55s spent in 
> > syntax-ppss, a more constructive approach would be to look into 
> > optimizing parse-partial-sexp. Or even give up on certain scenarios, 
> > admitting that waiting 55s once to visit the end of a 1 GB buffer is 
> > not so bad (and that could part could also be sped up by setting 
> > syntax-propertize-function to nil and using a very simple syntax 
> > table, for instance).
>
> It is bad, especially now that it became clear that in fact it's not 
> "waiting 55s once" but "waiting 55s each time the buffer is modified and 
> you move to another position in the buffer".

That was about a 1 GB buffer, right?

Let's take care of buffers with more reasonable sizes first, and then we 
can consider extremes. A separate threshold for syntax-ppss to avoid 
parsing the whole buffer might fit the bill.