Re: Finding out when a token is consumed

[Hans Aberg]

At 04:58 +0200 2003/05/12, Frank Heckenbach wrote:
>> In this variation, what you ask for is essentially a statement
>>     %preaction <code>
>> with <code executed right before any other action. It would probably not be
>> very difficult to supply Bison with such a feature, if somebody is willing
>> to supply it. (Essentially what is needed is putting <code> in a M4 macro,
>> and then insert that macro in the skeleton files right before the action
>> switch statement.)
>
>Perhaps a macro that takes 3 arguments (similar to YYLLOC_DEFAULT),
>the number of symbols and a reference to $$ and $1, so it can do
>something with the semantic values. (E.g., if someone prefers
>$$ = $N instead of $$ = $1 for the default action, this would be
>easy to do there.)

The whole point with the Bison use of M4 is that it should be easy to
change the skeleton files. So you can do whatever is convenient to you, and
if you find a olsition that is good, report it back here.

>> It would be nice with a feature that tells whether the lookahead is needed
>> or not, so one can implement sure context switches. While at tweaking the
>> skeleton files, there is a segment:
>>
>> yybackup:
>>
>>   // Try to make a decision without lookahead:
>>   n_ = pact_[state_];
>>   if (n_ == pact_ninf_)
>>     goto yydefault;
>>
>>   // Read a lookahead token.
>>   if (lookahead_ == empty_) {
>>     YYCDEBUG << "Reading a token: ";
>>     lex_();
>>   }
>>
>> Here you can clearly determine whether a lookahead is used or not. So if
>> you can find a good tweak for your purposes, it might be interesting for
>> you to report to Bug Bison how you did it so that it can be used as a
>> feature in Bison:
>>
>> I think it would be of general interest to get a better handling of this
>> lookahead problem.
>
>Unless I'm missing something, `yychar == YYEMPTY' should do this,
>doesn't it? I haven't tried this, except for the example in my
>original mail, but from the description in the manual (Action
>Features) I think it should work.

I only noticed that it should be possible to get the informatin out, but I
do not see how that could be made a convenient feature.

>However, that's only true for parsers with at most one token of
>look-ahead. OTOH, GRL can take an arbitrary amount of look-ahead
>(from the prespective of the parser it's only one look-ahead token,
>but to the semantic actions it appears like more because they're
>delayed). So I guess this method generally won't work there.

I have not looked at it carefully, but Bison manual suggests that the GRL
that Bison supplies is just LALR(1) with some additional branching for
parsing ambiguities.

Even though GLR is interesting, it is not as important for writing
conventional computer languages. So it will still be most important to
first handle the conventional deterministic parsing, and then move ahead
from there.

>To sum it up, in case it was a little confusing: For an LALR(1)
>parser (and probably many other ones), something like %preaction
>should work, checking for look-ahead using `yychar == YYEMPTY'.

I think of the preaction suggestion as separate form any other feature that
allows one to check whether the lookahead has been used. I do not see how
this other feature might be implemented yet. This other feature should
probably be somewhere in the yybackup segment indicated above.

  Hans Aberg