Re: User Token Numbers

[Paul Eggert]

> From: Akim Demaille <address@hidden>
> Date: 10 Apr 2002 09:34:10 +0200

> I'm in favor of
> 
> %token FOO 256
> %token BAR 257
> 
> and Bison moves error into 258.

Yes, I think that's a valid extension too, if the user doesn't define
the token number for error.

> | > Should the token error be #define'd in the .h?
> | 
> | Unix yacc does not do that.  Like Bison, it puts `#define YYERRCODE
> | 256' into the .c file, but nothing in the .h file.  
> 
> Which I don't understand in the case of Bison, given that this guy is
> i. hard coded (i.e., if you %token error 2222 you still get if defined
> to 256

No, it's not defined to 256 by Unix yacc in that case; it's defined to
2222.  That is, y.tab.c contains `# define YYERRCODE 2222' and the
tables are renumbered accordingly.

> | With Unix yacc, if
> | you have `%token error 1000' in the input, this changes the .c #define
> | to be 1000 instead of 256, and it redoes the tables accordingly, but
> | the .h file stays the same.
> 
> Does it _use_ it?

Yes, the Unix yacc y.tab.c uses the macro YYERRCODE, as well as
renumbering its tables.

> As noted by Hans, providing a means for the scanner to report an error
> to the parser can be useful.  But then, the question is returning
> `undefined', or `error'.

Either way, it would be a valid POSIX extension I think.  POSIX says
that a lexer should not return error, but I guess Bison can define
what happens when it does.

> That still does not completely answer the question: you may read
> the first section (before %%), and then assign a number of error
> before reading the grammar.  This is still single-pass

No, because you have to read the whole first section before assigning
any token numbers.  By "single-pass" I meant an implementation that
assigns a token number for a token T immediately after reading the
%token declaration for T, and before reading the next %token
declaration.  I think that was the motivation for saying error is
token 256.