Re: How to add pseudo vector types

[Stephen Leake]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> On 7/17/21 3:16 AM, Eli Zaretskii wrote:
>>> You do need to be careful to not read the garbage data from the
>>> gap, but otherwise seeing stale or even inconsistent data from the
>>> parser thread shouldn't be an issue, since tree-sitter is supposed
>>> to be robust to bad parses.
>> 
>> What would be the purpose of calling the parser if we know in advance
>> it will fail when it gets to the "garbage" caused by async access to
>> the buffer text?
>
> It won't fail, will it? I thought this was the point of TS, that it
> would reuse the initial parse on the "good" parts in subsequent
> parses.

There are limits to the error recovery, and throwing garbage text at
it is likely to encounter those limits. wisi is even more robust, but I
still get "error recover fail" daily.

>> So I don't see how this could be done without some inter-locking.
>
> Yes, there probably need to be some care around the gap area. But
> that's what I was referring to re. "optimistic concurrency".
>
>> And what do you want the code which requested parsing do while the
>> parse thread runs?  The requesting code is in the main thread, so if
>> it just waits, you don't gain anything.
>
> You'd have the parser running continuously in the background, every
> time there is a change. 

> When a piece of code requests a parse it blocks and waits, but
> presumably for not too long because a very recent previous parse means
> that the blocking parse is fast.

If the parser is truly fast enough to keep up with typing, this does
make sense. Good error correction is slower than non-so-good error
correction, so there might be a trade-off here.

On the other hand, in the typical case of the user typing characters,
font-lock is triggered on every character, so the parser is effectively
synchronous, and the inter-thread communication is wasted time.

We need some metrics on a real implementation to decide this part of the
design.

>>> In fact, depending on how robust tree-sitter is, you might even be
>>> able to do the concurrency-control optimistically (parse everything
>>> up to close to the gap, check that the gap hasn't moved into the
>>> region that you read, and then resume reading or rollback).
>> 
>> I don't understand what you suggest here.  For starters, the gap could
>> move (assuming you are still talking about a separate thread that does
>> the parsing), and what do we do then?
>
> Nothing, we start the next parse when this one completes.

By "nothing", I think you mean "abort the parse".

>> Bottom line, I think what you are suggesting is premature
>> optimization: we don't yet know that we will need this. 
>
> I thought we knew that a full parse of some files could take over a
> second; 

Yes, for both tree-sitter and wisi. wisi can take even longer if lots of
error correction is required (I have a time-out set at 5 seconds). But
that happens when the file is first opened; I doubt any user would start
typing that fast. I know I typically take a while to just look at the
text, and then navigate to the point of interest.

> but yes, it will be nice if we can find a synchronous way to avoid
> having to do a full parse.

Hmm. "looking at the text" is better done after it is fontified, so
doing a faster but possibly worse parse and fontification on just the
initial visible region might be a good idea. 

While the partial parse is running, we could also spawn a parser thread
to run the full parse.

And if the user scrolls before the full parse is done, do a second
partial parse on the new visible region.

I'll put that on my list of things to try in wisi.

-- 
-- Stephe