Re: Finding out when a token is consumed

[Frank Heckenbach]

Hans Aberg wrote:

> At 20:38 +0200 2003/05/08, Frank Heckenbach wrote:
> >As a first attempt, I tried to handle them during lexing. But that's
> >too early when bison reads look-ahead tokens. E.g., when the input
> >is `1+2#+3', bison will read the second `+' before it can do the
> >first addition, so the directive will affect the first addition
> >which it shouldn't do.
> 
> Unless you try integrate this somehow into the grammar, such things are
> very difficult to handle with Bison as the generated parser (as you say)
> may or may not make use of the lookahead. The problem here is that you say
> that this stuff can be inserted anywhere, which is essentially saying that
> you do not have a language ruled by a grammar.

It is ruled by a grammar -- but this grammer is not nice (as shown
in my first example), so I try to avoid doing it this way.

However, that's nothing unusual, I think. Many grammars bison is
used for are not really context free (identifier names etc.). But
that's no real obstacle -- one just checks these things in the
semantic actions, i.e. outside of the parser.

Even hairier things are possible with lexical tie-ins (as described
in the Bison manual!). (BTW, the example given there could also be
described by a grammar, by lexing `a1b' as a special token which is
"an identifier or a hex number" and which is accepted in both
contexts in the grammer, doing some trickery with the semantic
value; maybe it will be necessary to duplicate all rules and
nonterminals that are part of expressions; perhaps it will even
create some conflicts. It's possible, sure, but whether it's nice to
do in practice, doubtful ...)

In a theoretical sense, it might be cleaner to put all such things
in a proper grammar (even if it causes some conflicts and might even
require a more general and less efficient parsing method). In
practice, it's probably better to use an efficient LALR(1) parser
for the most part, and do the ugly bits outside of the parser. My
question was only about how to interface it to the parser in the
least painful way ...

> You might experiment with grammars like:
>
> [...]

Uhm, well, this is just what I tried in the examples I sent (t1.y
and t2.y) and which had the problems I described.

I don't think this discussion is going anywhere since I seem to be
repeating myself with the examples etc. Obviously, the answer to my
initial question is no. Thanks for your time, anyway.

Frank

-- 
Frank Heckenbach, address@hidden
http://fjf.gnu.de/
GnuPG and PGP keys: http://fjf.gnu.de/plan (7977168E)