Re: A vision for multiple major modes: some design notes

[Alan Mackenzie]

Hello, Eli.

On Sun, Apr 24, 2016 at 09:29:58AM +0300, Eli Zaretskii wrote:
> > Date: Sat, 23 Apr 2016 21:08:07 +0000
> > Cc: address@hidden, address@hidden
> > From: Alan Mackenzie <address@hidden>

> > I see three layers of software, here:  Major modes, super modes, and
> > subsystems.  What is the relationship of each of them to islands?

> > Super modes essentially deal with islands - that is what their main
> > purpose is.  They create islands, they destroy them, possibly they
> > coalesce them, they coordinate the rare interactions between islands
> > (yanking for example), they coordinate change hooks as they affect
> > islands.  Most of the changes I have proposed is in features directly to
> > support super modes' handling of islands.

> > Subsystems code, like redisplay, font locking, timers, ...., is going to
> > have to deal with islands incidentally - that is not its main purpose,
> > but there is no getting away from it.  A redisplay action might act on
> > several islands, so might a font locking action.  And so on.

> > But major modes?  The abstraction I propose is that major modes see their
> > own parts of the buffer as the entire buffer, and know nothing of
> > islands or gaps between them.  This is a clean abstraction and will lead
> > to all the advantages enumerated a few paragraphs back.

> > Eli, you seem to disagree with the above analysis.  Would you like to
> > outline your scheme of abstractions on this topic?

> Most of my comments were not about the abstractions.  I don't have any
> alternative scheme to offer, because I have no experience in using,
> let alone writing, multiple modes in the same buffer.

But you have undoubtedly suffered the frustration of bits of scripts,
possibly html files, and the like, not being properly fontified/indented
for lack of such multiple modes.

> > You say that extensive changes will be needed to support multiple modes
> > in a buffer, and this is clearly true.  Where we seem to differ is where
> > these changes should be made.  I want the vast bulk of these changes to
> > be in super mode support and subsystems.  You seem additionally to want
> > to make subtantial changes in the major mode "layer".  I cannot see this
> > as a good thing at the moment.

> I'm saying that worrying about the amount of changes in major modes at
> this stage is premature optimization.

You seem to be saying I should abandon "abstraction A" (that major modes
should remain unaware of islands) as a design principle.  Without this
principle, I'm not sure how much of my design notes make any sense.  I
certainly have no idea of what to replace it by.

> If major modes will have to adapt themselves in non-trivial ways, e.g.
> by changing their regexps or font-lock settings, it's not a big deal.

How do you know?  What I foresee happening is a lot of island handling
code being duplicated many times over, over many major modes.  I think
that is a big deal.

> It is much more important to make sure the design doesn't contradict
> more basic assumptions and design principles of Emacs, including the
> low-level code which implements searching, syntax, redisplay, etc.,
> because if the contradiction does happen, you will at best have a bunch
> of hairy problems to solve, and at worst will simply fail to produce a
> workable solution.

The very basic assumption that each buffer has exactly one major mode is
being superseded.  That is bound to have repercussions on several other
assumptions which are dependent on it, including in the ones you
identify.  Searching, syntax, redisply, etc., will all need to be adapted
because that basic assumption (one major mode) will no longer hold.  The
challenge is to identify all the code that implicitly assumes that
assumption.

I think some of these other dependent assumptions will become ambiguous.
For example, at the moment BEG and Z point to the start and end of the
part of the buffer the current major mode administers (this being the
entire buffer).  Nobody up till now has bothered to separate those two
meanings of BEG and Z.  Such disambiguation will be necessary to support
multiple major modes.  I've already proposed doing this by means of the
magic variable `in-islands'.

> IOW, I suggest to forget for a while about the amount of changes major
> modes will need, and leave that for later.  At this stage, you should
> be worried much more about how core design features of Emacs will work
> with islands, and make sure you have all that figured out, before you
> decide that the island design is valid.

I have spent quite some time studying data.c, syntax.c, xdisp.c,
buffer.[ch], lots of font locking code, and likely quite a few other
relevant files.  I haven't come across anything that would be difficult
to adapt for the island mechanism - just there's a lot to adapt.

> In practice, this means that, for example, I would expect you to study
> all the uses of search in the low-level code, before you decide that
> making [:space:] match an island edge is sound.

[ Actually, it's the entire island I'm proposing be matched as WS. ]

I tend to approach it from the other direction: is that handling of an
island as whitespace a satisfactory abstraction or not?  If it is, the
code will follow.  If it's not, attempts to apply it will collapse in
confusion, probably quite quickly.

> E.g., did you know that even bidi.c, which is about as low-level as you
> can get, uses regexp search to look for a certain combination of
> whitespace characters?

No, but it doesn't surprise me.

> Did you consider how this will work when islands are in the way?

Yes.  The bulk of the adaptation to bidi.c will be the generic changes in
search.c, etc., so that the bidi.c regexps will continue to work despite
the text it's matching over being two islands with a gap in the middle.

I know little about bidi, but there might have to be design decisions
made about how it should behave when the text it's dealing with isn't
contiguous in the whole buffer.

> What about basic features like find_newline -- did you look into that?
> You see, if any of these break due to islands, you have some major
> rewrites on your hands, and the ripples will probably be very
> far-reaching.  The need to change major modes pales by comparison.

No, I hadn't looked at find_newline.  But it will need looking at
regardless of whether a space in a regexp matches an island.  At the very
least, it will have to behave differently for finding newlines in an
island chain rather than finding them in the whole buffer.

-- 
Alan Mackenzie (Nuremberg, Germany).