Re: locked narrowing in ELisp

[Eli Zaretskii]

> Date: Fri, 19 Aug 2022 02:10:49 +0300
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 18.08.2022 09:25, Eli Zaretskii wrote:
> 
> >> I guess I didn't understand your concern, sorry. Invisible is handled
> >> somewhere on the high level, OK. I did not mistake it for 'intangible'.
> > 
> > The concern is that some parts of the display code will ignore the
> > invisible text and some won't.  The display code doesn't expect
> > BEGV..ZV restriction to behave that way, it expects these limits to
> > affect every part of Emacs.
> 
> That's not a fundamental problem. I believe it's possible to come 99.9% 
> close (and even provide better usability, say, by highlighting the 
> narrowing bounds to indicate that there is more to the buffer, but it's 
> hidden), but if not, the implementation can go a little lower level:
> 
> The redisplay could repeat the trick I showed with Isearch and wrap most 
> of its logic in (with-soft-narrow ...) -- which translates the soft 
> narrowing into an actual one temporarily. Then the 'invisible' property 
> won't even be required.

I think you still underestimate the magnitude of the problem and its
fundamental nature.  The display engine is not a closed subsystem, it
uses a huge number of Emacs infrastructure code from other subsystems:
buffers, text properties, overlays, markers, character composition,
regular expressions, direct access to buffer text, etc.  All of those
places are also used from other commands and functions, outside of
redisplay.  You cannot "wrap [their] logic in (with-soft-narrow ...)",
because their logic is used in many more use cases, and there's no way
such low-level code can distinguish between the cases, nor should it
(because it will make it too slow).

> > That would mean all of the commands.  A lot of changes (or a lot of
> > breakage).  It doesn't sound like a good solution, at least not better
> > than just biting the bullet and introducing 2 different kinds of
> > narrowing with some ways of telling Emacs internals which of the two
> > to obey at any particular moment.
> 
> Two kinds of narrowings is inherently a more complex solution.

Some problems don't have simple solutions.  The narrowing is a very
fundamental feature in Emacs, it permeates all of our code to the
lowest levels.  Such fundamental features cannot be side-stepped by
tricks and hacks.  The change must be in those same basic levels.

> > (The latter is actually the tricky
> > part, IMO.)
> 
> If we just make the "locking" feature to be accessible to Lisp, the 
> majority of the code shouldn't care or be aware of it. Just some 
> particular features/functions/packages will make use of the locking 
> argument, limiting the 'widen' calls in the rest of the Emacs from 
> widening past the locked region.

I'm not talking about locking!  I'm talking about the basic
requirement to distinguish between "user-level" narrowing and the
other kind.  How do you tell the low-level stuff, such as
set_marker_restricted, which of the two narrowing kinds to apply?
That function is used from every place where we set window-point and
window-start positions (as well as many others), so almost any code
that wants to move one of those will eventually call it.  Commands do
that, but redisplay and other places do it as well.

This is the tricky part of introducing two kinds of narrowing; the
rest is very simple and basically mechanical.

> > Making a specific command ignore narrowing is easy.  Your proposal
> > implicitly assumes that the number of commands that want to ignore
> > narrowing is much larger than the other kind.
> 
> That is indeed the key assumption. I'm inclined to believe that not only 
> it is larger, but that the set of commands that should ignore user-level 
> narrowing also grows faster.

What about commands that apply narrowing as part of their internal
operation -- is that "user-level" narrowing or the other kind?  E.g.,
some/many commands that operate on the region when it's active begin
by narrowing to that region -- which kind is that?

> With that in mind, creating any precise statistics doesn't seem 
> possible, even if one decides to try.

If that's so, then any assumptions about which set is significantly
larger cannot be safely made, either.  We should proceed without any
such assumption, i.e. without adopting any solutions which would be
based on such assumptions and will break if the assumptions are proved
false.

> On the flip side, though, it doesn't seem like diff-hl needs any support 
> for user-level narrowing (ignoring it is fine), and company-mode "just 
> worked" when I tested it with soft-narrow-to-region.
> 
> Same goes for all other packages I maintain.

IME, it is not a good idea to base such fundamental design decisions
on a couple of user commands and their needs.

> > Moreover,
> > I think it might make sense for some commands to honor or ignore the
> > narrowing depending on the use case, and that is not solved with your
> > proposal.
> 
> Doesn't seem difficult:
> 
> (if something
>      (with-soft-narrow
>       do-the-thing)
>    do-the-thing)
> 
> or without the macro:
> 
> (save-restriction
>    (when something
>      (let ((bounds (soft-narrow-bounds)))
>        (and bounds
>             (narrow-to-region (car bounds) (cdr bounds)))))
>    do-the-thing)

We are mis-communicating: the problem is not how to use basic
conditionals in Emacs Lisp, the problem is how you design and
implement that "something", such that we won't need to rewrite every
single command out there.