Re: Bug #25608 and the comment-cache branch

[Stefan Monnier]

> I also have an attachment to it because it works, and would save me
                                                 ^^^
                                               for you

> demoralizing work debugging bugs caused by open parens in column zero in
> comments.

Is that really the only reason?  It seems like an amazingly complicated
way to go about it.  Let's see some alternatives:
- set open-paren-in-column-0-is-defun-start to nil.
- add a font-lock rule which highlights open-paren-in-column-0 inside
  comments in bright red.
- use my syntax-ppss-based patch.

> Bug #22983 is a flaw.

Great!  We're trying to have a reasoned argument; I tell you that using
this term to describe this problem is not helping and you insist on
using it.  From where I stand, this qualifies as provocation.

> Also the cache invalidation in syntax-ppss is less than rigorous.

Yup, syntax-ppss's implementation is not perfect.  That can be improved.

> For example, the cache isn't invalidated when syntax-table text
> properties are applied or removed.

This is not a correct characterization of the most common
cache-invalidation problem with syntax-ppss: there's a correlation
between the problem and syntax-table text properties, but that's all: it
also affects all other properties, but it doesn't affect all changes to
the syntax-table text properties.

>> BTW, your comment-cache doesn't fix that "flaw" and hence won't help any
>> of those users of syntax-ppss which can't be changed to use your
>> comment-cache.
> That's incoherent.  comment-cache was never intended to help those other
> uses, though it appears it could do so for most of them.

It's only incoherent if you refuse to see the larger picture.

> Can't be done, as I keep telling you.  comment-cache is solely for
> handling literals.

Then it's useless, AFAIC:
- we need syntax-ppss's data for lots of things.
- your code can't replace all those uses.
- so we're stuck with syntax-ppss, no matter how much you think it sucks.
- then we might as well use it in back_comment instead of your code,
  since it's there and is cheap.

> The question of "widening" is not difficult.  Narrowing a buffer should
> not change the syntax of the characters in it.  Doing so leads to
> inconsistencies.

I can agree with that.  But currently that's not how Emacs behaves, so
it's an incompatible change (which I'm quite willing to make, BTW), and
needs to come with some way to recover the other behavior when that one
is needed.

> If I understand correctly, the problem is that multiple-major-mode modes
> are trying to use narrowing to get a null syntactic context.

That's the typical example, but not the only one.

> They are trying this because we don't provide anything better.
> We should provide something better.

Agreed.

> I suggested such a something last spring ("islands").  If each buffer
> position has an unambiguous syntactic context the question of
> "widening" simply evaporates.

I think this is too specialized to a particular application (multiple
major modes).  We also need to accommodate other cases.  For that we
need to provide something equivalent to

    (save-restriction
      (narrow-to-region BEG END)
      ...)

but where syntax-ppss and friends will know that we shouldn't widen past
BEG/END and that BEG should be taken as "the (temporary) beginning of
the buffer".  Let's call it `with-region-as-subbuffer`.  Most likely,
this new functionality should also make it possible to temporarily
provide a different syntax-table.  Such things are used in various
circumstances where the author simply wants to reuse Emacs's syntax.c
code to avoid writing some ad-hoc parsing.

IOW, we need to provide something on top of which we can build this
`with-region-as-subbuffer` macro as well as your islands.


        Stefan