locked narrowing in ELisp

[Stefan Monnier]

The new "fast long-lines" code uses a "locked narrowing" to try and
avoid being defeated by naive font-lock (and other) code.

This is a good heuristic to try and make sure Emacs stays responsive
even in the presence of naive packages not designed for such long lines.

Beside the need to override this in those rare packages where we can
provide better behavior with careful coding, the new code also lacks
a way to install such a locked narrowing from ELisp, even though it
seems only natural to use similar heuristics from ELisp code (e.g. when
jit-lock is run from a timer).

But as pointed out by Lars, this is related to the old discussion about
"classifying" narrowing, so as to distinguish narrowings installed by
the user via `C-x n n` from those installed by packages like MMM-mode or
Info-mode (or nowadays those installed by the redisplay).
These distinctions are important because various code may need to look
"outside" of those narrowings in some circumstances, but which "outside"
to use depends on the particular situation.

We've discussed it in the past and I still don't know what would be
a good API, but one particular class of `widen` which is known to be
used within narrowings that should *not* be overridden is those `widen`
calls used in "parsers" that try to understand the context of
an operation.

Those `widen` will sometimes be used in circumstances where they may
actually want to *narrow* rather than widen (e.g. the narrowing might be
needed to make sure the parser only looks at the part of the buffer that
it can understand (e.g. in MMM-mode), or it might be needed in order to
avoid the risk of spending an eternity parsing GBs of text (at the cost
of misparsing in too-large buffers)).

Last time we discussed these issues, we ended up deciding that
indentation (like font-lock before it) should "eagerly widen" (i.e. undo
any user-setup narrowing) before calling the mode-specific code.
This way, the indentation and font-lock code of a major mode should
simply never need to call `widen` and things like the MMM-mode (or the
long-lines code in redisplay) can just setup the narrowing before
calling that major mode code without any need to "label" the narrowing.

This was an elegant solution, but it seems to be too limited, because
the parsing code that want(ed) to `widen` is typically also called from
other parts of the major mode than the indentation or font-lock code.

In the past, we played with having a new widening function (there was
a `prog-widen` briefly in `prog-mode.el`).  Maybe we should
(re)introduce such a thing (or maybe an optional arg to `widen` which
would indicate what "kind" of widening we want to do), coupled with some
kind of `widen-function` which MMM-mode and the redisplay could
setup to filter/tweak those widening.

Within an MMM-mode or "redisplay locked narrowing", we *generally* want
`widen` not to widen past the limits imposed by MMM or by the redisplay,
but there will be cases where we do want to allow it (e.g. in
`nlinum-mode` the widening will usually want to be allowed to override
those narrowings in order to compute the right line number), so we
clearly not all calls to `widen` are the same.


        Stefan