Re: Syntax tables for multiple modes [was: bug#22983: syntax-ppss returns wrong result.]

[Dmitry Gutov]

On 03/21/2016 03:05 AM, Vitalie Spinu wrote:
> > > On Sun, Mar 20 2016 17:58, Dmitry Gutov wrote:
>
> > On 03/20/2016 02:15 PM, Vitalie Spinu wrote:
>
> > IIRC, using first-column is fairly justified, the outer mode can't add extra
> > indentation to the submode is a reliable, sane way
>
> The inner mode cannot often make that decision either.

What decision? One case where the mode cannot return its proposed 
indentation at all, is when the resulting column would be negative. 
Using first-column can make it positive again via simple addition.

Using calculate-indent-function which returns a numeric value, would 
solve that as well, of course, at the expense of having to update all 
major modes out there, and documentation. And making whatever 
third-party guides are out there obsolete in this regard. I'm not really 
against that, mind you.

> Same inner mode can be
> used in very different multi-mode contexts, each with their own semantics for
> chunks/headers/indentation. Reducing all that to a simple (first-column
> . previous-chunk) pair and letting inner mode do the job is surely not
> enough. The only actor to make that decision should be multi-mode engine itself.

I'm not claiming that using previous-chunk is good.

> Instead of teaching modes about multi-modes, a much better idea is to introduce
> `calculate-indent-function` which would accept POS and optional STRING-AFTER and
> STRING-BEFORE. This function will return the indentation of STRING-AFTER at POS
> assuming there is a virtual STRING-BEFORE just before POS.

Strings? Indentation engines do not deal with strings, they deal with 
buffer contents. Having them handle this possibility would also amount 
to sharing a part of multi-mode logic.

Instead, if you want to know what indentation an inner mode would return 
if STRING-BEFORE was before it, insert that string into the buffer 
(while inhibiting undo history). Call the indentation function, then 
remove the string. Any performance concerns with that?

> Most modes indent reliably
> based on one previous line,

Ruby doesn't. Most modes based on SMIE will need more than the previous 
line in the general case, too.

> Then a lot of modes don't even care about what's in the current line, so
> STRING-AFTER will be irrelevant as well.

Almost all of them care whether the current line contains }, or `end', 
or `else', and so on.

> > > It's essentially a half-backed implementation of "hard widening" discussed
> > > earlier. Why not impose the widening restriction directly in `widen` then?
> > > Maybe bring widen to elisp and rename C widen into widen-internal. Then add
> > > generic `prog-hard-widen-limits` which would be checked along
> > > prog-indentation-context limits.
>
> > Right! At the very least, I we should extract the second element of
> > prog-indentation-context into a separate variable, and make prog-widen more
> > prominent.
>
> Not sure about removing second element. Good thing about keeping all of them in
> one place is for the indentation engine to be concerned with a single variable.

Didn't you mention font-lock and syntax-propertize yourself? Why would 
they call a function that's solely dependent on an indentation variable?

In any case, your hard-narrowing proposal is very similar. Surely you 
don't want to keep the second element of prog-indentation-context after 
hard-narrowing becomes available?

> Only consumers of `hard-widen-limits` should be concerned with its side
> effects. But that's uniformly better than current situation when you cannot do
> much about restricting widen.

OK, so *every* consumer of widen will have to obey the hard limits. That 
might work, if there's no low-level code that absolutely has to always 
be able to widen to the whole buffer.

> BTW, I parse-partial-sexp must abide hard-widen-limits as well.

If you want parse-partial-sexp to obey limits, you narrow the buffer 
around it.

> This way the
> request aired in bug#22983 of parse-partial-sexp == syntax-ppss will be
> automatically satisfied. You won't need syntax-ppss-dont-widen either.

That doesn't seem relevant. That bug is about stale cache values between 
different narrowing bounds.

> A patch that would require hunting every single mode out there and implementing
> multi-modes locally should have been more carefully considered IMO. Emacs 25 is
> not yet there, so it's not late to reconsider that decision.

I concur.