Re: User Token Numbers

[Akim Demaille]

| > From: Akim Demaille <address@hidden>
| > Date: 09 Apr 2002 11:59:42 +0200
| > 
| > the user _has_ to write
| > 
| > %token error 257
| > %token FOO   256
| 
| Yes, that's my reading of it, if they want it to be portable to all
| POSIX implementations.  Admittedly the standard doesn't contain this
| particular example.
| 
| I think it's OK for Bison to accept
| 
|   %token FOO   256
|   %token error 257
| 
| as an extension.  I don't think a diagnostic is required when
| accepting the extension.

The extension I have in mind is considering error an internal symbol
(but visible, of course) as $undefined, i.e., Bison has to adjust
itself to the user.  In other words, I'm in favor of

%token FOO 256
%token BAR 257

and Bison moves error into 258.  I fail to see the point of reserving
256.  Yet, IMHO, reserving 0 for EOF instead of -1 what not a real
good idea.


| > Should the token error be #define'd in the .h?
| 
| Unix yacc does not do that.  Like Bison, it puts `#define YYERRCODE
| 256' into the .c file, but nothing in the .h file.  

Which I don't understand in the case of Bison, given that this guy is
i. hard coded (i.e., if you %token error 2222 you still get if defined
to 256), and ii., it's not used at all.

src/bison-exp/data % fgrep 256 bison.simple                      nostromo Err 1
#define YYERRCODE       256
src/bison-exp/data % fgrep ERRCOD bison.simple                    nostromo 9:26
#define YYERRCODE       256

| With Unix yacc, if
| you have `%token error 1000' in the input, this changes the .c #define
| to be 1000 instead of 256, and it redoes the tables accordingly, but
| the .h file stays the same.

Does it _use_ it?

| The lexical analyzer is not supposed to return the error token, so
| that probably explains why historically the token was not in the .h
| file.  The standard does not explicitly prohibit putting `#define
| YYERRCODE whatever' into the .h file, so I guess it'd be OK to do that
| if there's a good reason to.

As noted by Hans, providing a means for the scanner to report an error
to the parser can be useful.  But then, the question is returning
`undefined', or `error'.

| > Why does it matter to map error to 256?
| 
| My guess is that it's so that this:
| 
| %token FOO   257
| 
| will work even on single-pass implementations of yacc.  Otherwise, a
| single-pass implementation that by default mapped error to 257 would
| reject `%token FOO 257'.  Historically, grammars just used numbers
| greater than 256, and my guess is that the POSIX authors wanted such
| grammars to work on single-pass implementationms.

That still does not completely answer the question: you may read
the first section (before %%), and then assign a number of error
before reading the grammar.  This is still single-pass, but the code
of error can change.

Anyway, the bottom line seems to be that it is an arbitrary decision,
not something motivated.