Re: Atox problem solved (I think)

[Hans Aberg]

At 21:45 +0200 2004/05/23, Magnus Lie Hetland wrote:
>I have come to the conclusion that the best way to go is to do as I
>planned originally -- to transform the relevant grammars by inserting
>"wildcard" sections that are simply sequences of tokens that cannot
>begin any legal productions at that point.

This is surely the right way to go, as you then more easily can add more
advanced grammar features later. In case you are still looking for the
parser tweaking problem, the problem we earlier discussed was this:

Suppose one has a parser for a simple precedence calculator grammar, where
one is allowed to insert debugging tokens into the expression. Then, if the
expression
    a + b * c
is modified into
    a + b t_1 ... t_k * c
by debugging tokens t_1, ..., t_k, one would want their token actions to be
executed after the expression a + b has been computed. But Bison will here
scan ahead for the token * before making the decision whether to compute a
+ b. Thus, the tokens t_1, ..., t_k must be piped, and their actions would
be computed at the time state containing the token * is put onto the stack.

This can then be done by tweaking the "Read a lookahead token" segment in
the parser code:
  // Backup.
yybackup:
  // Try to make a decision without lookahead:
  n_ = pact_[state_];
  if (n_ == pact_ninf_)
    goto yydefault;

  // Read a lookahead token.
  if (lookahead_ == empty_) {
    YYCDEBUG << "Reading a token: ";
    lex_();
  }
Here, before lex_() is called, the pipe should be cleared of debugging
tokens, and their token actions should be executed. Then, in a loop, each
lex_() token should be examined and piped, if a debuggning token. WHen the
first non-debugging token is found that one should be used as the new
lookahead token.

Your situations seems to be similar, only that you merely want the
debugging tokens to be merely discarded.

  Hans Aberg