bug#61285: (Sometimes very) slow font-lock after %w in ruby-ts-mode

[Dmitry Gutov]

On 06/02/2023 02:08, Yuan Fu wrote:
> Interesting. Perhaps it has to do with how tree-sitter implements the
> "incremental" part of the parser? But the profile doesn’t look like it’s
> spending time parsing, 

According to my tests, what gets slower are the treesit-query-capture 
calls. And I mean all of them (for every element in 
treesit-font-lock-settings), not just the first one, which I imagine 
would be the case if tree-sitter needed to finish parsing the current 
buffer contents.

If I just wrap the treesit-query-capture calls inside 
treesit-font-lock-fontify-region in benchmark-progn, with %w inside the 
'if' block the queries are an order of a magnitude faster than with it 
at top level.

E.g. in the ruby.rb example, the former look like

...
Elapsed time: 0.001648s
Elapsed time: 0.001498s
Elapsed time: 0.001211s
Elapsed time: 0.000949s
Elapsed time: 0.000950s
...

and the latter are like

...
Elapsed time: 0.006567s
Elapsed time: 0.006583s
Elapsed time: 0.007072s
Elapsed time: 0.006867s
Elapsed time: 0.006575s
Elapsed time: 0.006608s
...

Multiply that by 19 (the number of rules), and we get the perceived delay.

And for associations.rb, the query times are 0.004322s vs 1.083029s.

> I need to look at what does
> ts_tree_cursor_current_status actually do (maybe it’s used in parsing?)

  // Private - Get various facts about the current node that are needed
  // when executing tree queries.
  void ts_tree_cursor_current_status(

https://github.com/tree-sitter/tree-sitter/blob/84c1c6a271cd0ab742ce0f46cd3576a6f6bf5b8c/lib/src/tree_cursor.c#L284

I see it is called by ts_query_cursor__advance, which is in turn called 
by ts_query_cursor_next_match. And ts_query_cursor_next_capture, which 
we don't seem to be using.

Here's an existing report on its tracker which might be relevant: 
https://github.com/tree-sitter/tree-sitter/issues/1972

Similar perf report, though not exactly the same.