bug#56393: Actually fix the long lines display bug

[Eli Zaretskii]

> Date: Wed, 06 Jul 2022 14:41:08 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: larsi@gnus.org, 56393@debbugs.gnu.org
> 
> > I think we must, because font-lock is run by the display engine, and if 
> > font-lock makes redisplay significantly slower, it's part of the same 
> > problem.
> 
> I disagree.  It's not the redisplay part that make font-lock slower, but 
> how it parses the buffer.  So it's a different problem.

How is it different from the code in the display engine that calls
next-single-char-property-change or remove-text-properties, or loops
over all the overlays at certain position calling overlay-get?  In
Emacs nowadays font-lock is almost always called as part of redisplay,
so I don't see how we can separate them and say that this is a
different problem.

> > Oh, we agree here: it's definitely way better.  I'm just saying that a 
> > complete solution cannot force users to make such sacrifices.
> 
> Users can reasonably expect and understand that in exceptional 
> circumstances a non-essential editing feature is, by default, turned off, 
> if the purpose of doing that is to maintain responsiveness.  If you load a 
> very big file in a browser (or for that matter in any application), it 
> will become sluggish.  Try to open the 1GB hugedictionay.json file with 
> vim, nano, VS Code, Atom, and try opening it with the patched Emacs.
> 
> (Atom is the only one that displays a warning: "Atom will be unresponsive 
> during the loading of very large files. Do you still want to load this 
> file?"  If you proceed, Atom becomes unuseable.)

I invite you to look at this from a different aspect angle.  Your idea
is basically to present to the display engine a small portion of the
buffer, whose size is 30K characters.  If you actually cut 30K
characters from any of the files we are using for this work, do you
see Emacs choking on any of them if you don't turn off font-lock and
leave 'recenter' as it behaves on master?  If not, then why doesn't
the same work with auto-narrow-mode?

And if a 30K-character cut is still clunky, then maybe 30K is not the
right default value?  For example, why not use 2 window-fulls before
and after the window (assuming that gives us a smaller chunk)?

The point is that we should try to squeeze the most out of this
narrowing idea, before we start disabling up features.  Because
disabling features is a kind of retreat, an indication that we turned
every stone and couldn't make Emacs fast enough, so we are kinda
giving up.  And it's too early to give up, IMO.

> > Why is 'recenter' at all relevant, btw?  It isn't called by the display 
> > code, AFAIR, so why do we have to disable it?
> 
> Because commands in auto-narrow-widen-automatically might call "recenter" 
> for aesthetic purposes.  One example is end-of-buffer.  Which means that 
> what you have is (progn (widen) (recenter)), which takes a few seconds 
> without fundamentally changing what is on display.

That's what I thought.  And to me this means that you see an example
of a problem I mentioned earlier: code from the display engine is used
in commands that basically have nothing to do with redisplay per se,
and your current implementation doesn't take care of those calls into
the display code by commands like recenter, C-v, C-n, etc.  I think
sooner or later we will need to present a narrowed buffer to them as
well.

> > I think there's a misunderstanding here: what these Lisp programs don't 
> > expect is to find narrowing that is different from what the user sees or 
> > expects top have.  Narrowing is largely a user-level feature, so using 
> > it for internal purposes is problematic at best.
> >
> 
> Sorry, I don't get what you mean.  Do you have a concrete example of a 
> problematic function?

What kind of example do you want to see?  What do you mean by
"problematic" in this context?

> A Lisp program that takes into account the possibility that the user uses 
> narrowing and that needs to have access to the whole buffer will use 
> (save-restriction (widen) ...).  A Lisp program that requires the user to 
> narrow the buffer to work properly will still work, because explicit 
> narrowing can still be used with auto-narrow mode.  What are the remaining 
> possibilities?

When the user narrows the buffer, he/she doesn't expect the parts
outside the restriction to be accessible.  But when the user didn't
narrow, the results of having some part of a command being able to
access only 30K around point _will_ surprise users.

Which is why I'm saying that it is better to restrict the display code
via some other means, and leave the narrowing alone.  After all, it is
our code that deliberately references BEGV and ZV in the display
engine, so using some other values shouldn't be that difficult, I
think.