Re: A vision for multiple major modes: some design notes

[Eli Zaretskii]

> Date: Sun, 24 Apr 2016 16:57:21 +0000
> Cc: address@hidden, address@hidden
> From: Alan Mackenzie <address@hidden>
> 
> > Most of my comments were not about the abstractions.  I don't have any
> > alternative scheme to offer, because I have no experience in using,
> > let alone writing, multiple modes in the same buffer.
> 
> But you have undoubtedly suffered the frustration of bits of scripts,
> possibly html files, and the like, not being properly fontified/indented
> for lack of such multiple modes.

No, not really.

> You seem to be saying I should abandon "abstraction A" (that major modes
> should remain unaware of islands) as a design principle.

It's not an abstraction, it's a design goal.  And yes, I think you
need to forget about it for a while.

> Without this principle, I'm not sure how much of my design notes
> make any sense.

I don't see how that invalidates your proposal.

> I certainly have no idea of what to replace it by.

I suggest replacing it with nothing.  Minimizing changes in major
modes (and elsewhere) is a simple economy principle; you don't need to
worry about us forgetting it.

> > If major modes will have to adapt themselves in non-trivial ways, e.g.
> > by changing their regexps or font-lock settings, it's not a big deal.
> 
> How do you know?  What I foresee happening is a lot of island handling
> code being duplicated many times over, over many major modes.  I think
> that is a big deal.

If it is, we will cross that bridge when we get to it.

> > It is much more important to make sure the design doesn't contradict
> > more basic assumptions and design principles of Emacs, including the
> > low-level code which implements searching, syntax, redisplay, etc.,
> > because if the contradiction does happen, you will at best have a bunch
> > of hairy problems to solve, and at worst will simply fail to produce a
> > workable solution.
> 
> The very basic assumption that each buffer has exactly one major mode is
> being superseded.  That is bound to have repercussions on several other
> assumptions which are dependent on it, including in the ones you
> identify.  Searching, syntax, redisply, etc., will all need to be adapted
> because that basic assumption (one major mode) will no longer hold.  The
> challenge is to identify all the code that implicitly assumes that
> assumption.

There's exactly zero references to major mode in C sources.  (There's
a function to store the major mode in the corresponding slot of the
buffer object, but I see no code looking that slot's value.)  And for
a good reason: the major mode is an entirely Lisp-land phenomenon, it
does all of its work by setting local variables and hook functions.

So I think your assumption that having more than one mode in a buffer
is already a cataclysm is incorrect.

> I think some of these other dependent assumptions will become ambiguous.
> For example, at the moment BEG and Z point to the start and end of the
> part of the buffer the current major mode administers (this being the
> entire buffer).  Nobody up till now has bothered to separate those two
> meanings of BEG and Z.  Such disambiguation will be necessary to support
> multiple major modes.  I've already proposed doing this by means of the
> magic variable `in-islands'.

Indeed, I'm much more worried by the effect of islands in BEGV and ZV
than by the fact that there could be more than one major mode active.
Unlike references to the major mode, the number of places that use
BEGV and ZV is enormous, and the unwritten assumptions about them are
abundant and well entrenched.

> I have spent quite some time studying data.c, syntax.c, xdisp.c,
> buffer.[ch], lots of font locking code, and likely quite a few other
> relevant files.  I haven't come across anything that would be difficult
> to adapt for the island mechanism - just there's a lot to adapt.

We should try to minimize that impact as much as we can.

> I tend to approach it from the other direction: is that handling of an
> island as whitespace a satisfactory abstraction or not?

It's not an abstraction at all.  It's a trick, a device to make
adaptation to the island-world easier.  That text between two islands
of the same chain should be invisible for the mode that's active in
the chain -- that is an abstraction.  But no one says that text must
be treated as whitespace -- this is simply a convenient means to reach
your ends.  However, other means towards the same end might be
available, onces that don't overload [:space:] with an entirely alien
meaning.

> > Did you consider how [bidi.c search] will work when islands are in the way?
> 
> Yes.  The bulk of the adaptation to bidi.c will be the generic changes in
> search.c, etc., so that the bidi.c regexps will continue to work despite
> the text it's matching over being two islands with a gap in the middle.

Doesn't that contradict your design to limit point-min and point-max,
since redisplay must be island-aware?

> I know little about bidi, but there might have to be design decisions
> made about how it should behave when the text it's dealing with isn't
> contiguous in the whole buffer.

These design decisions should predate the island-as-whitespace
discussion, IMNSHO.  And if you are sure this feature cannot happen
without affecting bidi.c and search.c, then yes, you should study
those and understand what they do and how.

> No, I hadn't looked at find_newline.  But it will need looking at
> regardless of whether a space in a regexp matches an island.  At the very
> least, it will have to behave differently for finding newlines in an
> island chain rather than finding them in the whole buffer.

See, the ripples are starting already.  That is why I say we should
try to find a design that doesn't require rethinking, redesigning, and
rewriting every single piece of our infrastructure.  If we don't, we
are making the implementation and testing of this feature a much more
complex and hard job than it must be.