bug#60650: 30.0.50; tree-sitter: parsing (and font locking) occasionally breaks during editing

[Mickey Petersen]

Yuan Fu <casouri@gmail.com> writes:

> Mickey Petersen <mickey@masteringemacs.org> writes:
>
>> It is occasionally possible to put the tree-sitter parser, I am
>> guessing, into an erroneous state w.r.t. the contents of the buffer it
>> is charged with parsing.
>>
>> This rarely presents itself, and when it does, it only seems to happen
>> when text is deleted and inserted in short order (`delete-region' +
>> `insert') from an elisp command.
>>
>> The resulting erroneous state then breaks font locking, resulting in 
>> mis-fontified code.
>>
>> I must note that the parse tree is _correct_ and not invalid. There is
>> no easy way to 'fix' this bug, as newlines and other manual edits
>> rarely fixes the issue. There is also no easy way to insist that
>> tree-sitter must re-do everything. Presumably deleting and recreating
>> the parser could work as a workaround? I do not know.
>>
>> The only way to truly and accurately fix the issue is to save and
>> re-open the file. This then forces a complete re-parse of the buffer
>> and fontification, fixing the issue.
>
> What do you mean by "correct and not invalid"? If a complete re-parse
> fixes the issue, then the old parse tree must be incorrect, right? As
> for mis-fontified code, is the position wrong (everything skew left or
> right for a few character), or there is no fontification, or something
> else? If the position is wrong, then it’s probably a bug in syncing the
> parser’s position wrt the buffer content.

I mean that the source code is syntactically correct. Given a clean
slate, the parser would not report any erroneous nodes.

But, somehow, both the font lock and indentation remains incorrect, even when I 
type text that would otherwise force a reparse. The only way to fix this is to 
save the buffer (in a correct state), close and re-open the file. That fixes it.

That, and undoing the edited steps done by the elisp code. That snaps it back 
to its original state, and the parse tree is back to normal.

It's... weird.

>
>> I have zero means of reliably of reproducing this, unfortunately: it is 
>> sporadic, intermittent, and happens somewhat infrequently.
>>
>> So I'd love some ideas on how to go about debugging this.
>
> I think the first step is to better understand what’s happening. Some
> screenshots are also welcome. Thanks for your effort in advance!
>

Yep. Will do!


> Yuan