bug#60650: 30.0.50; tree-sitter: parsing (and font locking) occasionally breaks during editing

[Yuan Fu]

Mickey Petersen <mickey@masteringemacs.org> writes:

> It is occasionally possible to put the tree-sitter parser, I am
> guessing, into an erroneous state w.r.t. the contents of the buffer it
> is charged with parsing.
>
> This rarely presents itself, and when it does, it only seems to happen
> when text is deleted and inserted in short order (`delete-region' +
> `insert') from an elisp command.
>
> The resulting erroneous state then breaks font locking, resulting in 
> mis-fontified code.
>
> I must note that the parse tree is _correct_ and not invalid. There is
> no easy way to 'fix' this bug, as newlines and other manual edits
> rarely fixes the issue. There is also no easy way to insist that
> tree-sitter must re-do everything. Presumably deleting and recreating
> the parser could work as a workaround? I do not know.
>
> The only way to truly and accurately fix the issue is to save and
> re-open the file. This then forces a complete re-parse of the buffer
> and fontification, fixing the issue.

What do you mean by "correct and not invalid"? If a complete re-parse
fixes the issue, then the old parse tree must be incorrect, right? As
for mis-fontified code, is the position wrong (everything skew left or
right for a few character), or there is no fontification, or something
else? If the position is wrong, then it’s probably a bug in syncing the
parser’s position wrt the buffer content.

> I have zero means of reliably of reproducing this, unfortunately: it is 
> sporadic, intermittent, and happens somewhat infrequently.
>
> So I'd love some ideas on how to go about debugging this.

I think the first step is to better understand what’s happening. Some
screenshots are also welcome. Thanks for your effort in advance!

Yuan