bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 06.08.2022 09:07, Eli Zaretskii wrote:
> > Date: Fri, 5 Aug 2022 23:23:25 +0300
> > Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> > From: Dmitry Gutov <dgutov@yandex.ru>
> >
> > > We need to look at more than just M->.  C-n/C-p, C-v/M-v, C-l are also
> > > important, as are the time it takes from typing M-x or M-: until you
> > > see the prompt in the minibuffer, and the time to update the display
> > > after inserting or deleting a single character.
> >
> > I'm not seeing any particular sluggishness in these operations when
> > visiting dictionary.json.
>
> Numbers, please.  You have a very fast machine, so what doesn't look
> sluggish on your system could very well be so on others.

How do you measure these operations including the redisplay lag?

Anyway, all of these look instant.

> > > Thanks, so I guess we may have a solution for JSON files, if disabling
> > > syntax-propertize-function doesn't have any downsides.  What about
> > > other modes that we see in files with long lines, like XML?
> >
> > Someone will need to test it with some typical large file. xml-mode
> > (alias to nxml-mode) does have a syntax-propertize-function, but it's
> > probably faster than js-syntax-propertize.
> >
> > > And how scalable is the solution you propose, i.e. how it behaves in
> > > JSON files with a much longer lines?
> >
> > parse-partial-sexp is O(length of text span)
>
> Isn't that the part you proposed to remove?  I was asking about the
> remaining parts.

No, removing syntax-propertize-function just made parse-partial-sexp 5x 
faster here, resulting in much faster fontification near EOB.

But in a large enough buffer, it's still going to be the slowest part.

> > Meaning, it scales linearly. You'll see a 10x delay in a JSON file that
> > is 10x as large.
>
> Linear scaling is less optimal than O(0), which is what the current
> solution produces.

Correct syntax highlighting requires parsing the buffer from the 
beginning. Otherwise we get random results, essentially.

Just like depicted on my latest screenshot for downloadify.js: the 
beginning of the narrowed region ended up inside a string, and as a 
result several screenfuls were entirely mis-fontified, with strings and 
non-strings inverted.