bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode

[Dmitry Gutov]

On 21/01/2023 00:24, Yuan Fu wrote:
> > On Jan 19, 2023, at 10:28 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> >
> > Hi Yuan,
> >
> > On 18/01/2023 08:50, Yuan Fu wrote:
> > > > > > Should treesit--font-lock-fast-mode be locally bound inside that
> > > > > > function, so that it's reset between chunks? Or maybe the condition
> > > > > > for its enabling should be tweaked? E.g. I don't think there are any
> > > > > > particularly large or deep nodes in ruby.rb's parse tree. It's a
> > > > > > very shallow file.
> > > >
> > > > Yeah that is a not-very-clever hack. I’ve got an idea: I can add a C
> > > > function that checks the maximum depth of a parse tree and the maximum
> > > > node span, and turn on the fast-mode if the depth is too large or a node
> > > > is too wide. And we do that check once before doing any fontification.
> > > >
> > > > I’ll report back once I add it.
> > > I wrote that function. But I didn’t end up using it. Instead I added a
> > > "grace count", so that the query time has to be longer than the
> > > threshold 5 times before we switch on the fast mode instead of 1.
> > > My main worry is that simply looking at the parse tree would not catch
> > > all the case where there will be expensive queries.
> >
> > That might be true, but a criterion that doesn't specify conditions exactly can 
> > give no guarantee against false positives.
>
> The condition is “query is (consistently) slow”, that’s why I thought measuring 
> the time is the most direct way.

The benchmark itself might be artificial, in that it's measuring the 
font-lock of a specific buffer, in whole, for 1000 iterations. But Juri 
must have come up with the original report based on real usage scenario.

OTOH, the scenario which it might correspond to, is used typing in the 
same buffer for a long time (triggering thousands of refontifications, 
possibly partial ones). I don't know if it's feasible to try to 
reproduce it specifically. But, again, anything that can happen once can 
happen 4 more times.

> > > Could you try the latest commit and see if the fast mode still switches
> > > on when it shouldn’t?
> >
> > At first it seemed to help, but then I switched the major mode a couple more times, and 
> > ran the benchmark twice more, and the "fast mode" switched on again.
> >
> > Which seems to make sense: there is no resetting the counter, right?
> >
> > So if previously it happened once somehow during a certain scenario, now I have 
> > to repeat the same scenario 4 times, and the condition is met.
>
> I was hoping that the scenario only happen once, oh well :-) I’ll change the 
> decision based on analyzing the tree’s dimension: too deep or too wide 
> activates the fast mode. Let’s see how it works.

Thank you, let me know when it's time to test again.