bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

Hello, Dmitry.

On Mon, May 16, 2016 at 12:50:54AM +0300, Dmitry Gutov wrote:
> On 11/12/2015 08:54 PM, Alan Mackenzie wrote:

> > The fix to bug #16247 meant no longer setting syntax-begin-function to a
> > non-nil value.  This is the condition which used to cause the appropriate
> > font-lock-keywords form to get added to lisp-font-lock-keywords-1/2.  It
> > no longer is.

> Looking into this, I'm not sure we still want to highlight them. The 
> aforementioned bug, now fixed, mirrored the justifications that we give 
> in the manual and the comments for the highlighting of parens in the 0th 
> column:

> "The convention speeds up many Emacs operations, which would otherwise 
> have to scan back to the beginning of the buffer to analyze the syntax 
> of the code."

Note this convention is still active.

> and

> ;; Try to detect when a string or comment contains something that
> ;; looks like a defun and would thus confuse font-lock.

> We don't have to scan back to the beginning of the buffer, we can use 
> syntax-ppss (and it's more reliable with bug#16247 fixed).

Sorry, this isn't true.  The scanning back to BOB is done at the C
level, in function back_comment.  syntax-ppss isn't suitable for use
here (Stefan's view, not merely mine), because syntax-ppss doesn't react
to changes in the syntax table, and suchlike.

> font-lock doesn't get confused by something looking like a defun inside 
> a docstring (try it; I wasn't able to get it highlight something wrong).

You might be getting confused, here.  The scanning back to BOB which is
slow doesn't just happen in font lock; it can (and does) happen
anywhere.  It's just font lock's job to warn the user about this, so
that she can correct it by adding in a backslash, for example.

Things do get confused, for example see bug #22884, where there was an
open paren in column zero in our own C sources.

> M-x beginning-of-defun does get confused, though. If *that* is problem 
> what we want to detect, .....

Not particularly.  We want the user to be warned about things
potentially going wrong in back_comment, and anything which calls it.
The problem we want to fix is the lack of font-lock-warning-face on
these parens in column 0.  Anything beyond that is not for Emacs 25.1.

> .... I think the patch should look like this:

> diff --git a/lisp/font-lock.el b/lisp/font-lock.el
> index 8ee9f69..eed2766 100644
> --- a/lisp/font-lock.el
> +++ b/lisp/font-lock.el
> @@ -1786,13 +1786,10 @@ font-lock-compile-keywords
>         (cons t (cons keywords
>                       (mapcar #'font-lock-compile-keyword keywords))))
>       (if (and (not syntactic-keywords)
> -          (let ((beg-function syntax-begin-function))
> -            (or (eq beg-function 'beginning-of-defun)
> -                   (if (symbolp beg-function)
> -                       (get beg-function 'font-lock-syntax-paren-check))))
> -          (not beginning-of-defun-function))
> +             (not beginning-of-defun-function)
> +             open-paren-in-column-0-is-defun-start)

No.  open-paren-in-column-0-is-defun-start is a variable that the user
can change at any time.  We can't make our font-locking dependent upon
what its value was at some time in the past.  If open-paren-... belongs
anywhere, it's in the form just beyond the end of your patch's text.

Do you understand the consequences of taking out the check on
syntax-begin-function?  (I certainly don't.)  It would be good if Stefan
could express a view, here.

>       ;; Try to detect when a string or comment contains something that
> -     ;; looks like a defun and would thus confuse font-lock.
> +     ;; looks like a defun and would thus confuse beginning-of-defun.

Also no.  It's more general than that.  I think "would thus confuse
Emacs" would be more accurate.

>       (nconc keywords
>              `((,(if defun-prompt-regexp
>                      (concat "^\\(?:" defun-prompt-regexp "\\)?\\s(")

-- 
Alan Mackenzie (Nuremberg, Germany).