bug#56682: locked narrowing

[Eli Zaretskii]

> Date: Wed, 30 Nov 2022 22:09:59 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: 56682@debbugs.gnu.org, monnier@iro.umontreal.ca, dgutov@yandex.ru
> 
> I had to refresh my memory about this, and I think there is a 
> misunderstanding here: we check whether a buffer contains long lines only 
> when it is about to be redisplayed and its contents have changed enough 
> since the last time it was displayed.  What happened between these two 
> redisplays can be anything, e.g. it could be a buffer that has not been on 
> display during an hour and that has been filled by a process in the 
> meantime.  Using BEGV/ZV is wrong, again because we don't know what 
> happened between these two redisplays, it could have been filled by a 
> process and narrowed, and if after switching to that buffer the user 
> widens it, Emacs would become unresponsive.  Or it could be a buffer that 
> has been emptied in the background by a command, filled with the contents 
> of a file, and narrowed to some portion of that file (e.g. a single hunk 
> of a diff file); again if the user widens it to look at the whole file, 
> Emacs will hang.  The only safe way to be certain that a buffer does not 
> contain long lines is to check the whole buffer.

I still don't understand why we need to look beyond the current narrowing:
we will never display anything outside of that.  Why should we care about
what happens in parts of the buffer that we don't display, and thus cannot
affect redisplay performance?

And again, there are technical problems with doing that without a call to
'widen' -- this is why I made that change: it caused trouble in some
situation that I no longer recall.  If you want to argue for looking at the
entire buffer, you will have to insert a call to 'widen' there, otherwise
the code which does that will not be safe.

And in any case, the BEG vs BEGV issue is secondary here.  My main worry is
valid for buffers without any narrowing at all -- in fact, especially for
those.  So let's leave the BEGV issue alone, or discuss it separately.  here
it is just a tangent.

> > If we are going to narrow the buffer to PT ± N characters, then 
> > searching inside slightly more than this region (say, twice more or 5 
> > times more) should be enough, I think.  It might fail in some borderline 
> > cases, but most files with very long lines have almost all of their 
> > lines long, so I see no problem here.
> 
> The intention when this detection mechanism was designed was explicitly to 
> also support non-file visiting buffers, e.g. a shell buffer in which a 
> user cat's a file with long lines.

This takes a minor and insignificant aspect of the idea I presented and
ignores the main part.  Please respond to the main ideas, not to inaccurate
wording.

And most buffers with very long lines in real life do come from files, not
from shell commands.  So even here you take a relatively rare use case and
make it more significant than it is.

> > OTOH, "punishing" large buffers that have no long lines at all doesn't 
> > strike me as a good balance.  Your proposed patch will solve a 
> > particular use case, but it cannot solve all of them.
> 
> Indeed, by definition no heuristic can solve all problems.

I'm proposing what should be a better heuristic.

> > When modiff becomes large enough, we will search again, and the 
> > threshold of 8 is not so large to prevent it from happening frequently 
> > enough to be an annoyance.
> 
> Please, let's not throw away something that was designed with care and 
> about which literally no one complained since it was introduced, for the 
> sole reason that an artificial benchmark shows a slowdown of ~80 
> milliseconds in a 30 MB buffer.

That benchmark is an example of many use cases that can happen in real life,
in a large buffer with no long lines and a lot of editing activity.  I'm
surprised you don't see it.

> And note that it's not "modiff becomes large enough", it is "modiff
> changed enough between two redisplays".

Once again, a useless argument about a problem with inaccurate wording.  Do
you really think I don't understand how modiff works in this case?  Why
waste energy and bandwidth on such tangents?  Let's focus on the main issue
at hand.

The main issue at hand is how to avoid needless scanning of the entire
buffer for long lines, something which doesn't scale well with buffer size.
For very large buffers without any long lines, this is a regression in Emacs
29, because Emacs 28 didn't do that.  IOW, we make "good" buffers suffer
because of the (rare) danger presented by potentially "bad" buffers.  We
need to look for ways of making this degradation as small and as rare as
possible.

> > That annoyance will be for no good reason in buffers that have no long 
> > lines.
> 
> It's not "for no good reason", it's for an excellent reason: it's the 
> price to pay to ensure that Emacs remains responsive in as many cases as 
> possible.

It will be very hard to explain this "price" to users who never have to deal
with very long lines.  We must look for every way of making the price lower.

> And it's not an annoyance, for all basic editing commands the detection
> loop will not run.

It will run whenever redisplay discovers that a window needs an update on
display, and there was enough changes in buffer text since last redisplay.
It should be clear that with low enough threshold for text changes, this
could happen quite frequently.

> > If you are still unhappy with such a simple heuristics, we could go with 
> > something slightly more complex, like change the threshold of modiff 
> > dynamically based on the buffer size and perhaps the number of times we 
> > already searched without finding long lines -- the larger these two are, 
> > the higher we could bump the threshold.
> 
> I don't see how such a heuristic could catch a case such as a user cat'ing 
> a file with long lines in a shell buffer.

The modiff value will change by a large delta, because it basically counts
the number of characters inserted since last redisplay, right?  So the
difference

   CHARS_MODIFF - UNCHANGED_MODIFIED

will change by a large value, and will exceed even a higher threshold.
AFAIU, this should solve the case you present here.