bug#62951: 29.0.90; c-ts-mode: Incorrect fontification due to FOR_EACH_TAIL_SAFE

[Yuan Fu]

> On Apr 23, 2023, at 2:04 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 23/04/2023 03:28, Yuan Fu wrote:
>> What do you think of extending the parser to support these macros instead? 
>> (So we fork tree-sitter-c.) If we can fix the parser, we don’t need to 
>> retrofit hacks onto font-lock, indent, etc, separately, and it truly fixes 
>> the problem. The downside is compiling from grammar source to grammar.c 
>> needs rust and node tools. But I guess depending on the grammar maintained 
>> by tree-sitter’s author isn’t too much different from depending on the 
>> grammar maintained by another individual (ie, me)?
> 
> We had also talked at some point about replacing the actual text that the 
> parser sees with something else.
> 
> If this can be done in a straightforward way (with tracking the subsequent 
> correspondence of "real" text back to the buffer for syntax highlighting), 
> that might be the perfect solution: we'd have a defcustom which would hold a 
> list of macros used in the current codebase in the form of templates, and 
> we'd set a bunch of them in emacs/.dir-locals.el.
> 
> I'm not sure how difficult this is to implement and maintain, but it's 
> probably going to be less work to maintain than a fork of the grammar.

Sounds to me a bit difficult to write. Eg, translating between tree-sitter 
position and buffer position efficiently isn’t too easy. Now plus narrowing, 
and what if the narrowing boundary is in the middle of a replace region?

My idea right now is to use the range feature in tree-sitter. Since the “body” 
of FOR_EACH_TAIL is valid C, I can either set the ranges for the parser so it 
ignores FOR_EACH_TAIL, or I can add another parser that only parses the body of 
FOR_EACH_TAIL.

Yuan