bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 05.08.2022 15:20, Gregory Heytings wrote:
> > Like, we're going from a 255 (?) second delay to 2 second delay 
> > already without breaking fontification. And yet you're eager to go 
> > from 2 seconds down to ~0 and sacrifice highlighting correctness?
>
> Yes.  Because as I told you your 2 seconds are 4 for me and 8 for Eli 
> and 20 for someone else.  And that's in a relatively small file.

Is it a "relatively small file" if we've effectively been unable to edit 
such files for all of 40 years of Emacs's existence?

> Note that if it were 2/4/8/20 seconds once, and then no further 
> slowdowns while editing the file, that would perhaps be okay.  But 
> that's not the case, you will regularly see a similar 2/4/8/20 seconds 
> delay.

Do our users regularly edit 30MB files? And do a lot of changes in them? 
In different areas?

Note that, again, to see the same delay you would have to edit that file 
near the beginning, and then visit its end again.

If I did that, though, I'm not sure whether I would be more 
inconvenienced by performance, or by broken syntax highlighting and sexp 
navigation. font-lock is not just eye candy: it also assists you when 
editing code.

For instance, if I were to edit dictionary.json, I might have needed to 
look for a certain key and change it somewhere. But if the said key is 
highlighted as a part of a string value in some places, and only as a 
key in some others, that can look and feel very puzzling, and slow down 
my work just the same.

Similarly, if I'm editing a large JSON file, I might want to write a 
small Lisp program which searches for a word, checks that it's inside a 
string (or, conversely, outside and thus looks like a key), makes all 
the necessary changes in an automated fashion, and saves the buffer. A 
broken syntax-ppss wouldn't let me do that.

Finally, yes, for some buffer size the initial wait is going to be too 
much. But that can have a separate solution with a separate threshold.