bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov]

On 05.08.2022 18:33, Eli Zaretskii wrote:
> > Date: Fri, 5 Aug 2022 17:41:51 +0300
> > Cc:monnier@iro.umontreal.ca,56682@debbugs.gnu.org,gregory@heytings.org
> > From: Dmitry Gutov<dgutov@yandex.ru>
> >
> > > > And to address the idea of "unlocking" the narrowing: I think I have
> > > > demonstrated that the remaining slowdown can be caused purely by the
> > > > length of the buffer and how long 'parse-partial-sexp' takes to parse
> > > > it.
> > > No, you haven't demonstrated that.
> > Apply the patch for xdisp.c that I have sent previously (it will be at
> > the end of this email too) and recompile Emacs.
> >
> > Now try two different scenarios. 1,2a and 1,2b.
> >
> > 1. Visit dictionary.json. It will ask you whether to open such big file
> > literally, but after you answer 'y', it will display the beginning of
> > the file quickly.
> > 2a) Evaluate (benchmark 1 '(save-excursion (parse-partial-sexp 1
> > (point-max)))), note the reported delay.
> >
> > Kill and re-visit the file.
> >
> > 1. (same as before)
> > 2b) Press M->, note the delay you see.
> >
> > The delays in scenarios 1,2a and 1,2b should be ~the same. They are so
> > in my testing.
> >
> > Or try this scenario: 1,2a,2b. Step 2b should work instantly here.
> How is (parse-partial-sexp 1 (point-max)) related to the issue at
> hand?

Or perhaps I should answer this way:

We move to near EOB.
fontification-functions are called.

jit-lock
calls
(font-lock-fontify-region point-near-buffer-end (point-max))
which calls
font-lock-fontify-syntactically-region
which calls both
  (syntax-propertize (point-max))
  and
  (syntax-ppss point-near-buffer-end) -> and it calls parse-partial-sexp

syntax-propertize will also likely call syntax-ppss itself, probably 
through the major mode's syntax-propertize-function. But if 
syntax-propertize-function is nil, parse-partial-sexp gets called 
anyway, over the whole buffer, which makes it the main workload in 
fontifying near EOB.

Now, if syntax-propertize-function is non-nil, parse-partial-sexp will 
also call it, and it adds its overhead (sometimes a multiple of p-p-s), 
which also scales linearly with the length of the buffer.

So if one can demonstrate that (parse-partial-sexp (point-min) 
(point-max)) takes about the same time as it takes to fontify the last 
screen-ful of a buffer, then that says that everything else that 
jit-lock does to fontify, is negligible, time-wise.