bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode

[Yuan Fu]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 21/01/2023 00:24, Yuan Fu wrote:
>> 
>>> On Jan 19, 2023, at 10:28 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>>>
>>> Hi Yuan,
>>>
>>> On 18/01/2023 08:50, Yuan Fu wrote:
>>>>>>> Should treesit--font-lock-fast-mode be locally bound inside that
>>>>>>> function, so that it's reset between chunks? Or maybe the condition
>>>>>>> for its enabling should be tweaked? E.g. I don't think there are any
>>>>>>> particularly large or deep nodes in ruby.rb's parse tree. It's a
>>>>>>> very shallow file.
>>>>>
>>>>> Yeah that is a not-very-clever hack. I’ve got an idea: I can add a C
>>>>> function that checks the maximum depth of a parse tree and the maximum
>>>>> node span, and turn on the fast-mode if the depth is too large or a node
>>>>> is too wide. And we do that check once before doing any fontification.
>>>>>
>>>>> I’ll report back once I add it.
>>>> I wrote that function. But I didn’t end up using it. Instead I added a
>>>> "grace count", so that the query time has to be longer than the
>>>> threshold 5 times before we switch on the fast mode instead of 1.
>>>> My main worry is that simply looking at the parse tree would not catch
>>>> all the case where there will be expensive queries.
>>>
>>> That might be true, but a criterion that doesn't specify conditions exactly 
>>> can give no guarantee against false positives.
>> The condition is “query is (consistently) slow”, that’s why I
>> thought measuring the time is the most direct way.
>
> The benchmark itself might be artificial, in that it's measuring the
> font-lock of a specific buffer, in whole, for 1000 iterations. But
> Juri must have come up with the original report based on real usage
> scenario.
>
> OTOH, the scenario which it might correspond to, is used typing in the
> same buffer for a long time (triggering thousands of refontifications,
> possibly partial ones). I don't know if it's feasible to try to
> reproduce it specifically. But, again, anything that can happen once
> can happen 4 more times.
>
>>>> Could you try the latest commit and see if the fast mode still switches
>>>> on when it shouldn’t?
>>>
>>> At first it seemed to help, but then I switched the major mode a
>>> couple more times, and ran the benchmark twice more, and the "fast
>>> mode" switched on again.
>>>
>>> Which seems to make sense: there is no resetting the counter, right?
>>>
>>> So if previously it happened once somehow during a certain scenario, now I 
>>> have to repeat the same scenario 4 times, and the condition is met.
>> I was hoping that the scenario only happen once, oh well :-) I’ll
>> change the decision based on analyzing the tree’s dimension: too
>> deep or too wide activates the fast mode. Let’s see how it works.
>
> Thank you, let me know when it's time to test again.

Sorry for the delay. Now treesit-font-lock-fontify-region uses
treesit-subtree-stat to determine whether to enable the "fast mode". Now
it should be impossible to activate the fast mode on moderately sized
buffers.

Yuan