Re: Syntax propertize in font-latex (Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments after braces)

[Tassilo Horn]

Ikumi Keita <ikumi@ikumi.que.jp> writes:

>>> BTW, I noticed in the patch that `font-latex.el` appears to still
>>> make use of `font-lock-syntactic-keywords`.  This var has been
>>> obsolete since Emacs-24.1 and it's like to disappear in Emacs-29, so
>>> it would be good to fix this issue (maybe using
>>> `syntax-propertize-via-font-lock`).
>
>> Thanks for the heads-up, I've done that just now.
>
>>   
>> https://git.savannah.gnu.org/cgit/auctex.git/commit/?id=ab3bfaf1033f0d6e0ecfe3172794a6363450b207
>
> Unfortunately, this breaks regression test and results in:
> SUMMARY OF TEST RESULTS
> -----------------------
> Files examined: 13
> Ran 49 tests, 35 results as expected, 11 unexpected, 3 skipped
> 3 files contained unexpected results:
>   latex/latex-test.log
>   latex/font-latex-test.log
>   context/context-test.log

Oh, sorry. :-)

> I encountered the same problem when I tried similar approach before. The
> reasons of these errors is that
> 1. `indent-according-to-mode' requires that syntax propertization is
>    done.
> 2. `syntax-propertize-via-font-lock' needs font lock has been
>    initialized in that buffer.
> 3. In batch mode, tests run before font lock is initialized because it
>    doesn't enter command wait loop. For example,
> (with-temp-buffer
>   (LaTeX-mode)
>   syntax-propertize-function)
>    returns nil.

I've just tried adding a (font-lock-ensure) in some tests which fixes
them.

> Attached is my preliminary attempt to avoid the errors by
> circumventing `syntax-propertize-via-font-lock'. (Sorry, it involves
> some unrelated changes.) Though I haven't finished it yet, I expect it
> works basically and doesn't interfere with regression tests.

I'm not sure I've actually separated the unrelated changes from the fix
for the issue at hand but it looks much more complex, doesn't it?

IMO, that the tests fail is just an artifact of the tests running in
batch mode which will never happen in any real editing context.  So I
wouldn't want to jump though hoops to fix the tests.  Maybe the simplest
fix could be to add an advice

  (define-advice LaTeX-mode (:after ())
    (font-lock-ensure))

so that our tests can be sure font-lock has been done after LaTeX-mode
has been activated.

> The problems is that it requires some changes in entries in
> `font-latex-syntactic-keywords', sometimes rather non-trivial ones as
> seen in the patch. It's possible to do required changes in files
> distributed with AUCTeX, but if some user adds entries to
> `font-latex-syntactic-keywords' in their private styles, this
> incompatibility can be a non-easy problem.

Indeed, and therefore I'd like to avoid such a drastic change if
possible.

FWIW, the following patch fixes all tests.

Bye,
Tassilo

--8<---------------cut here---------------start------------->8---
3 files changed, 9 insertions(+)
tests/context/context-test.el  | 3 +++
tests/latex/font-latex-test.el | 3 +++
tests/latex/latex-test.el      | 3 +++

modified   tests/context/context-test.el
@@ -24,6 +24,9 @@
 (require 'ert)
 (require 'context)
 
+(define-advice ConTeXt-mode-common-initialization (:after ())
+  (font-lock-ensure))
+
 (AUCTeX-set-ert-path
  'ConTeXt-indent-test/in
  "context-indentation-in.tex"
modified   tests/latex/font-latex-test.el
@@ -27,6 +27,9 @@
 (defvar font-lock-beg)
 (defvar font-lock-end)
 
+(define-advice LaTeX-common-initialization (:after ())
+  (font-lock-ensure))
+
 (ert-deftest font-latex-three-dollars ()
   "Test three consecutive dollar is ignored."
   ;; When the function `font-latex-match-dollar-math' encounters three
modified   tests/latex/latex-test.el
@@ -24,6 +24,9 @@
 (require 'ert)
 (require 'latex)
 
+(define-advice LaTeX-common-initialization (:after ())
+  (font-lock-ensure))
+
 (AUCTeX-set-ert-path
  'LaTeX-indent-tabular-test/in
  "tabular-in.tex"
--8<---------------cut here---------------end--------------->8---