Re: jit-lock refontifies too much

[Stefan Monnier]

>>>>> "martin" == martin rudalics <address@hidden> writes:

> With contextual fontification turned on jit-lock exposes yet another
> inefficiency.  When changing text you trigger eventual execution of
> `jit-lock-context-fontify' which removes the `fontified' text property
> from the current and all subsequent lines in the buffer.  This has two
> unpleasant consequences:

> - After jit-lock-context-time seconds of Emacs idle time, redisplay will
>    refontify all lines below the current one in any window showing the
>    current buffer.

> - After jit-lock-stealth-time seconds of Emacs idle time stealth
>    fontification refontifies all subsequent lines in the current buffer.

This second part predates jit-lock-context-*.  The introduction of
jit-lock-context-* was meant mainly to distinguish the above two cases so
that the idle time that triggers one can be changed without affecting
the other.

> I suppose that the latter was the main cause for complaints about
> "stealth fontification GCing a lot" some months ago.  The ensuing
> discussion lead to changing the default value of jit-lock-stealth-time
> to 16 seconds.

The jit-lock-stealth-* is indeed the part of the code that has caused some
problems for some people (I myself have it disabled).  The introduction of
jit-lock-context-* is what has allowed jit-lock-stealth-time to be pushed
from 3s to 16s (while jit-lock-context-time is itself set to 0.5s instead of
3s).

> Contextual fontification is useful since it reminds me whenever I forget
> to close a string or comment.  But the context of subsequent lines
> changes at most every 100th character I type.

We agree.

> Hence in at least 99 out of 100 cases contextual fontification just slows
> down editing and wastes processor cycles.

In theory, that's true.  In practice, I've yet to come up with a case where
it's noticeable.  Have you actually perceived this slow down, or are
talking hypothetically?  What do you compare it against?  Remember that in
Emacs-21, the exact same wasteful redisplay took place, except that it took
place after 3s of idle-time rather than 0.5s.

> The problem could be resolved by using the following reasoning (please
> correct me if I'm wrong):

> - Syntactic context is correctly established via the face property
>    assigned to a character by font-lock.  As an example, font-lock
>    assigns a character the font-lock-string-face property iff that
>    character is within a string - according to the best knowledge of
>    syntax-ppss.

That's not quite true since the font-lock-string face may be used for
elements not recognized by the syntax-tables.  But the exceptions are very
rare anyway and probably don't affect your proposed solution.

> - The syntactic context of a character may change iff preceding text is
>    modified (I ignore syntax table switching here).

Actually, jit-lock-multiline ws introduced specifically because this is not
necessarily the case.  In Perl, if you change

            s{a}{b}ex;
into
            s{a}{b};

the syntactic context of `b' is changed.  Yes: Perl's syntax sucks.
But again, this shouldn't affect your proposed solution.

> - When text is modified, the syntactical context of subsequent
>    characters may change iff the context of the first character
>    following the modified text changes.

Now, this is the core reasoning behind your proposed solution.

> Hence, it seems sufficient to have jit-lock-fontify-now compare the face
> property of the first character following modified text before and after
> refontification.  If the property did not change, the modification could
> not possibly change the syntactic context of subsequent text.

I like your idea.  But I know it'll fail in a case that I use on
a regular basis: nested comments.

If you change

    (* (* this is a nested comment
          blabla *)
       end of comment *)

to

    (* this is a nested comment
          blabla *)
       end of comment *)

the char after your modification will still be on font-lock-comment face,
but the text "end of comment *)" needs to be refontified.  Wait, I don't
even need nested comments, just take C mode:

     /* sfsgf
        tryjjy */
to
     //* sfsgf
        tryjjy */

Although in your case jit-lock will round up the "change" to a whole line,
so the problem shouldn't bite you in the case of C.

The above problems can be "easily" addressed by changing your algorithm to
not look at the face property, but instead to look at the return value of
syntax-ppss.  If it hasn't changed, then we know the subsequent text doesn't
need refontification.

Hmm... except of course for things like sh-mode where

       cat - <<EOF
       blabla
       EOF
to
       cat - <<EO
       blabla
       EOF

won't work right.  But that case can be handled specially (the problem in
that case is that the font-lock-syntactic-* setting of sh-mode will
actually cause the syntax-table property on the final EOF to change, but
it's done inside font-lock so it's a buffer modification which font-lock
won't notice because inhibit-modification-hooks is non-nil: we can require
sh-mode to call a special function to notify font-lock that it's modified
some part of the buffer).

> There is one complication: When text has not been fontified yet, the
> comparison described above would always indicate a context change.  This
> would be inefficient in some cases, for example, when scrolling backward
> into unfontified text.  The patch of jit-lock below tries to avoid this
> by using an additional text property called "jit-lock-context".

Adding yet-another-text-property is a waste of precious CPU and
memory resources.  It makes your suggestion pretty dubious.  Luckily, using
syntax-ppss instead of faces should not suffer from this same initialization
problem.


        Stefan