Re: treesitter local parser: huge slowdown and memory usage in a long file

[Vincenzo Pupillo]

Thank you Yuan!
Yesterday I tried your patch with my php-ts-mode (with an without 
treesit-primary-parser) and works fine. 
Yesterday I pushed a commit that uses the new variable. 
When I work at home, php-ts-mode is the major-mode I use,
and is almost done (I just need to add in support for flymake). 
Next week I hope to submit it for inclusion in emacs. 
It also seems to me that it might also be useful for trying out various 
combinations of parser. 

Thanks
V.



In data martedì 4 giugno 2024 06:53:47 CEST, Yuan Fu ha scritto:
> 
> > On May 27, 2024, at 3:24 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> > 
> > On 28/05/2024 01:03, Yuan Fu wrote:
> > 
> >>> But if one operation just changes text in that range (keeping its length 
> >>> intact, e.g. capitalizing the whole region), and another does the same 
> >>> (back to lower case), then the combined range would remain 200..300.
> >>> 
> >>> Computing that might be difficult without having access to the kinds of 
> >>> changes are being done (does tree-sitter report those?). OTOH, most of 
> >>> the time the most important part is the position of the beginning of the 
> >>> changes (e.g. for syntax-ppss), and we could treat the rest of the buffer 
> >>> as invalidated…
> >> Oh you’re absolutely right, the range will be shifted by later edits in 
> >> the buffer. It’ll be hella hairy to keep track of all that—say the 
> >> previous changed range is (100 . 200), and user inserted 50 chars in 
> >> position 150, we need to account for that and update the range to (100 . 
> >> 250) before merging the new updated ranges with this one.
> >> So it seems the best way is really to move treesit--pre-redisplay entirely 
> >> into the primary parser’s notifier, WDYT?
> > 
> > Yep, that sounds easier. And the performance should be about the same, even 
> > if it'd have a bit extra overhead in those theoretical complex cases.
> > 
> 
> Ok, I pushed a commit to master that does just that. I tried with C’s block 
> comment, and php-ts-mode. Everything seems to work fine.
> 
> I also added treesit-primary-parser. This is supposed to be another 
> configuration variable that a major mode should set. I’ve encountered various 
> cases where knowing the primary parser (parser that parses the entire buffer 
> rather than just a subset of it) would be very helpful. 
> Treesit-primary-parser can be auto-guessed if major mode doesn’t set it, so 
> it shouldn’t break anything. I’d love to know yours and Stefan’s thoughts on 
> it.
> 
> Yuan
>