bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments after braces

[Stefan Monnier]

Ikumi Keita [2022-03-02 23:47:56] wrote:
> Hello Stefan, thanks for your response.
>>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> In which operation does the "fails scanning for balanced {...} pairs"
>> happen?
>
> That's described in the example file test.dtx provided in the original
> post by Markus; double-clicking by mouse:

Duh!  Sorry for being dense, I didn't actually read the example's text.
Can't believe it was staring at me!

> --- test.dtx ---
> % \begin{macro}{\test}^^A This is an official dtx-comment
> % This does not work. Try to double-click to the closing brace before the
> % comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
> % \end{macro}

So, IIUC the same kind of problem already happens "everywhere" in Emacs
in slightly less trivial cases, such as

    %% \somemacro{some
    %%            multiline arg}

where double clicking on the braces will not do what we want (and will
often signal that same "Unbalanced parantheses" error).

This doesn't imply the OP's problem is negligible, of course, but
I think it does reduce its severity.

It also means that the approach you propose of only changing the `face`
and not the syntax tables is probably good enough.

> I think that we can do that for `font-latex-doctex-^^A' and
> `doctex-font-lock-^^A'. I guess AUCTeX needed separate function because
> of compatibility for older emacsen including xemacs.
>
> On the other hand, AUCTeX still needs its own function
> `font-latex-doctex-syntactic-face-function' different from
> `doctex-font-lock-syntactic-face-function' because we use different
> faces for its return value.

Indeed, practical issue tend to get in the way.
BTW, I noticed in the patch that `font-latex.el` appears to still make
use of `font-lock-syntactic-keywords`.  This var has been obsolete
since Emacs-24.1 and it's like to disappear in Emacs-29, so it would be
good to fix this issue (maybe using `syntax-propertize-via-font-lock`).


        Stefan