bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments after braces

[Ikumi Keita]

Hello Markus and all developpers,

I'd like to discuss this issue again.

After reconsideration, I came to think that there is little point in
discriminating two kinds of comments, i.e. those after "%" and those
after "^^A", as long as AUCTeX is concerned. It requires complicated
operation in syntax propertization (`font-latex-doctex-^^A') whereas it
calls for unwanted troubles like this bug#35140 and comments written in
`font-latex-find-matching-close':
,----
|         ;; XXX: Do not look up syntax-table properties since they may
|         ;; be misleading, e.g. in the case of "{foo}^^A" where the
|         ;; closing brace gets a comment end syntax.
`----

Thus I'm inclined to an idea just to fontify "^^A" type comments with
normal comment face, not bothering with changing its syntax. Attached is
a patch that implements this idea.

What do others think about this?

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

patch
Description: tentative patch

>>>>> Ikumi Keita <ikumi@ikumi.que.jp> writes:
>>>>> Markus Kohm <komascript@gmx.info> writes:

>> The LaTeX package doc, that is usually used via class ltxdoc for the
>> documentation part of dtx-files, declares ^^A as comment character.
>> But auctex's doctex mode fails scanning for balanced {...} pairs, if
>> } is immediately followed by ^^A. You can test it, with this example file:

> The reason of this error is the syntax-table property put on the closing
> curly brace.  In order to achieve different fontification on the docTeX
> documentation and the comment after ^^A, the function
> `font-latex-doctex-^^A' puts the "comment ender" syntax on the closing
> "}", which hides the proper syntax "closing paren paired with {" and
> thus leads to the reported error.

> Actually, `font-latex-doctex-^^A' is essentially just a copy of
> `doctex-font-lock-^^A' in `tex-mode.el', so the emacs built-in
> doctex-mode shares exactly the same problem with respect to the given
> test file at double-clicking.

> It was hard to me to resolve this problem keeping the closing "}" having
> "comment ender" syntax, so my proposal is to borrow an idea used at
> fontifying the verbatim-like environments.  I.e, to shift the position
> of "comment ender" syntax 1 character rightward and put another "comment
> starter" syntax on the second, not the first, "^" of ^^A.  The wrong
> fontification which the first "^" receives is overwrote afterwards by an
> additional entry in `font-latex-doctex-keywords'.  See the attached
> patch.

> Strictly speaking, this patch sets the boundary of two different types
> of comments in "SOMETHING^^A" at wrong position, not between "G" and the
> first "^", but between the first and second "^".  However, the treatment
> of comments of AUCTeX is rather regexp centric and slight in syntax, I
> expect this change doesn't bring serious bad side effects at editing
> docTeX documents such as filling and indenting.

> How do you think about this idea?  Any comments are appreciated.

> Regards,
> Ikumi Keita

>> --- test.dtx ---
>> % \begin{macro}{\test}^^A This is an official dtx-comment
>> % This does not work. Try to double-click to the closing brace before the
>> % comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
>> % \end{macro}
>> %
>> % \begin{macro}{\test}%^^A This is a workaround for an official dtx-comment
>> % This does work, but the \% after the closing brace is not wanted.
>> % \end{macro}
>> % \endinput
>> % Local Variables:
>> % mode: doctex
>> % TeX-master: t
>> % End:
>> --- end of file ---