Re: smie-next-sexp vs associative operators

[Stephen Leake]

I just realized I used "transitive" in this post when I should have used
"associative"; I don't know why. I apologize; I've got a head cold, and
between the direct effects and the drugs, I seem to be not thinking
quite straight ...

Stephen Leake <address@hidden> writes:

> Stefan Monnier <address@hidden> writes:
>
>>> I'm indenting "accept LILLY".  It should be indented relative to
>>> "select".
>>
>> That's odd.  I'd expect it to be indented relative to "or".
>
> The reason is that's how "if/then/else" works:
>
> if A then
>    B;
>
> elsif C then
>    D;
>
> elsif 
>   E then
>    F;
>
> else
>   G;
>
> end if;
>
> Here the internal keywords "then", "elsif" are not transitive; "else" is
> transitive. Here we are assuming "E" is a complex multi-line expression,
> so we can't use it as an indentation reference; we have to get back to
> "if" or "elsif". We call (smie-backward-sexp "then") to do that; it
> stops at "if".
>
> In the analogous situation with "select or", (smie-backward-sexp "or")
> stops at the previous "or".
>
> Why should a select statement be different? I guess you'd say "because
> it is different" :). Still, I'd like one rule that handles both, and
> it's quite easy to achieve. The fewer special cases I have, the easier
> it is to maintain in the long run.
>
>>> But this is exactly like the "elsif" case the comments are talking
>>> about, so I think the code is broken.
>>
>> The comment is imprecise, it's meant for "if ... elsif ... end" where
>> the BNF said ("if" ... "elsif" ... "end")
>
> That structure matches this comment and code (as does the Ada "if"):
>
>            ;; If the new operator is not the last in the BNF rule,
>            ;; and is not associative, it's one of the inner operators
>            ;; (like the "in" in "let .. in .. end"), so keep looking.
>            ((not (smie--associative-p toklevels))
>             (push toklevels levels))
>
> However, what determines whether an operator is "transitive" in this
> SMIE sense is more subtle; it depends on what the surrounding
> non-terminals are. When the two non-terminals are the same, the operator
> is transitive. When they are different, it is not. 
>
> For Ada, we have:
>
> "if" expression "then" statements "elsif" expression "then" statements
> "end" "if"
>
> "declare" declarations "begin" statements "end"
>
> These keywords are all non-transitive, because they have different
> non-terminals to the right and left; declarations, expressions, and
> statements are all distinct. "else" above is transitive because it has
> 'statements' on both sides.
>
> "select" statements "or" statements "else" statements "end" "select"
>
> Here "or" and "else" are transitive, because they have 'statements' on
> both sides. In fact have the same levels; both are (25 25) (illustrating
> your earlier point that the operator table is not invertible in general).
>
> Note that the "elsif then" pair also has the same levels on the left and
> right - they form a "transitive pair". In this case, smie-next-sexp will
> skip as many as appear.
>
> Which suggests another way to state my core complaint; since
> smie-next-sexp will skip an arbitrary number of "transitive pairs", it
> should also skip an arbitrary number of "transitive singles". Or at
> least, provide an option to do so.
>
>> rather than ("if" ... list-of-elseif ... "end") and then
>> list-of-elseif defined as (list-of-elseif "elsif" list-of-elseif).
>
> adding the missing nonterminals (and remembering that in C-like
> languages there are no "statements", only "expressions"):
>
> ("if" expressions "elseif" expressions "elseif" expressions "end")
>
> These "elseif" are transitive. But again, it is the surrounding
> non-terminals that determine this. In a language that distinguishes
> between expressions and statements, "elsif" will never be transitive,
> but "elsif expression then" will be.
>
>> I should fix it, thank you for pointing it out.
>>
>> I.e. this rule is to avoid stopping at "elsif" when scanning from "if" to
>> "end" or from "end" to "if".  It will still stop when scanning from an
>> "elsif" and bumping into another "elsif".
>
> Hmm. Calling (smie-forward-sexp "if") on the above "if" statement gives
> the following toklevel sequence (obtained by setting a break in
> smie-next-sexp):
>
> ((184) 3)   if
> nil         A
> (3 25)      then
> nil         B
> (37 36)     ;
> (25 3)      elsif
> nil         C
> (3 25)      then
> nil         D
> (37 36)     ;
> (25 3)      elsif
> nil         E
> (3 25)      then
> nil         F
> (37 36)     ;
> (25 25)     else
> nil         G
> (37 36)     ;
> (25 127)    end
> (127 (174)) if
>
> Now I see; the checks for associative in smie-next-sexp handle the case
> where there is _one_ associative operator (here "else") in a statement.
>
> And with the equivalent sequence in a C-like language that doesn't
> have "then", it would stop at the second elsif. I don't see why Ada and
> C-like should be different in this.
>
> Since "else" will be transitive in most languages, and appear only once,
> that would be a better keyword to use as an example in the current
> smie-next-sexp code comments.
>
> And I would still prefer having more control over this, to unify the
> behavior of smie-next-sexp on Ada "if" and "select", and between Ada and
> C-like "elsif". Similarly for "+"; if I had kept "+" in the grammar, I
> would not want smie-next-sexp to stop at it. That choice should be up to
> the smie user, not dictated by smie.
>
> As usual, this conversation has been educational - I learned more about
> what "transitive" means in SMIE :).
>
> --
> -- Stephe

-- 
-- Stephe