Re: parser error recovery algorithm vs treesit indentation "blinking"

[Dmitry Gutov]

On 04/04/2023 15:01, John Yates wrote:
> On Mon, Apr 3, 2023 at 5:49 PM Stephen Leake
> <stephen_leake@stephe-leake.org>  wrote:
> > That's because the tree-sitter
> > algorithm does not insert symbols, it only skips them.
> Is this a fundamental architectural limitation of tree-sitter's parsing
> scheme?  Was it a design decision that trying insertions would be
> too costly?  Or is it an improvement that should be explored?
>
> Has this been discussed in the wider tree-sitter community?  I would
> be surprised if emacs is the first to encounter this weakness.

I think the answer is "it varies". E.g. this unfinished snippet actually 
parses without errors:

  int foo() {

Even creating a "virtual" closing brace node, 0 characters in length.

But insert this snippet twice (maybe with a different function name) - 
and you get errors in the parse tree.

So there is some mechanism for virtual insertion.

One could say that our indentation mechanism could be at fault here (a 
little), given that the first step is to find the node enclosing the 
current position of point. But the "virtual" closer is positioned to be 
at the end of the previous line.

It's a matter of perspective, which side the fault lies at: maybe the 
virtual closer should be positioned at EOB, then our code would work 
fine with this example. This would make a lot of sense from my POV.

But maybe we should adjust the logic in this particular case: when 
between point and EOB is only whitespace, it could look for nodes at the 
end of the last non-whitespace character, and consider that the current 
node. This will need a fair amount of testing, I think, to make sure we 
don't get false positives this way.

Here's a relevant discussion, but there's nothing about positions in 
there: https://github.com/tree-sitter/tree-sitter/issues/224