Re: Finding out when a token is consumed

[Frank Heckenbach]

Hans Aberg wrote:

> >However, that's nothing unusual, I think. Many grammars bison is
> >used for are not really context free (identifier names etc.). But
> >that's no real obstacle -- one just checks these things in the
> >semantic actions, i.e. outside of the parser.
> 
> The unusual is that we do not get to know if this language you are working
> with has a given grammar or whether it just has the ability to throw in
> directives anywhere, as you claim. In the latter case, the hope diminishes
> rapidly.

The latter. Of course, as I said, one can also describe this by a
grammar, even a context-free one, but it seems an ugly way to do
it.

> >Even hairier things are possible with lexical tie-ins (as described
> >in the Bison manual!). (BTW, the example given there could also be
> >described by a grammar, by lexing `a1b' as a special token which is
> >"an identifier or a hex number" and which is accepted in both
> >contexts in the grammer, doing some trickery with the semantic
> >value; maybe it will be necessary to duplicate all rules and
> >nonterminals that are part of expressions; perhaps it will even
> >create some conflicts. It's possible, sure, but whether it's nice to
> >do in practice, doubtful ...)
> 
> The difference here is that these are some localize, well defined, features
> which can be handled by special tricks.

Localized WRT the grammar rules, not necessarily WRT the input
(since expressions can be arbitrarily long). And since you mentioned
alternative parsing methods in your previous mail -- it will fail,
e.g., in a GLR parser.

> If you want to make use
> of lexer context switches, you must make sure they do not clash with parser
> lookaheads.

Yes, that's exactly my problem here.

> This your comment and the next...
> 
> >> You might experiment with grammars like:
> >>
> >> [...]
> >
> >Uhm, well, this is just what I tried in the examples I sent (t1.y
> >and t2.y) and which had the problems I described.
> 
> ...suggest that you are designing your own language where directives can be
> thrown in just anywhere.

I'm not designing the language, it was given to me. I'm just trying
various ways to implement it.

Frank

-- 
Frank Heckenbach, address@hidden
http://fjf.gnu.de/
GnuPG and PGP keys: http://fjf.gnu.de/plan (7977168E)