Re: perl mode color highlighting working badly

[Richard M. Stallman]

I finally had time to study the last couple of weeks' messages about
this.

    font-lock-syntactic-keywords is not really applied eagerly to the whole
    buffer, but only to the region between point-min and the currently visible
    area.

That is good enough, provided this is done in all the cases it needs
to be done.  But I have doubts about that.

Can you tell under what circumstances it is done?  As far as I can
see, font-lock-fontify-syntactic-keywords-region is called only from
font-lock-default-fontify-region.  That seems to mean it can fail to
be called when it needs to be called.

Martin's mail seems to confirm that.

    With jit-lock the syntactic keyword pass is executed only till the end
    of the window showing the highest so far displayed position of the
    buffer.  This means that a parsing function investigating text after
    that position cannot rely on syntactic properties of the underlying
    text.  Since jit-lock is turned on by default the manual should warn
    about this.

This is called "syntactic fontification", but it really has nothing
to do with font-lock mode.  It is support for syntax parsing.

    `font-lock-fontify-syntactic-keywords-region' does that already.  But
    for languages relying on syntax properties one would have to call that
    function before every scan-sexps, parse-partial-sexp, skip-syntax ...

For correct behavior, every time syntax parsing wants to look at
some text, it should ensure that syntax property application has
been done up through that text.  This should happen whether or
not Font Lock mode is enabled.

You wrote:

    Indeed.  I've mentioned already a few times that the functionality offered
    by font-lock-syntactic-keywords should be added to syntax.el (which means
    make it independent from font-lock) after which font-lock-syntactic-keywords
    can be made obsolete.

Would you like to implement that now?

    And of course, this new functionality should ideally be a bit more
    flexible/powerful than font-lock-syntactic-keywords so that the job done by
    perl-font-lock-syntactic-face-function can be conveniently done there.

For right now, we don't NEED to make it more powerful.
That can be a second step later on.

    `jit-lock-context-*' just resets the fontified property from `start'
    until `point-max'.  If `start' is before `window-start' redisplay won't
    bother - the syntactic keyword pass will begin at `window-start' anyway.
    Syntax properties between `start' and `window-start' are not assigned.
    Hence, jit-lock guarantees syntax-properties only between `window-start'
    and `window-end'.

The new code in syntax.el needs to detect buffer changes, and force
rescanning of what follows.