[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug-bison] Bug in string-valued terminals

From: Tom Roberts
Subject: [bug-bison] Bug in string-valued terminals
Date: Sat, 25 Dec 2010 22:54:27 -0600
User-agent: Thunderbird (Macintosh/20080421)

I am using Mac OS X 10.5.8 (Leopard) with XCode 3.1.3; bison is:
    $ bison --version
    bison (GNU Bison) 2.3
I'm pretty sure this bug is independent of OS or environment. I have not built bison 2.4.3, but its bison.info still has the bug, and its code appears to.

I want to have the grammar define the keywords as literal strings, so on first call my yylex() will build up its list of keywords by scanning yytname[] for entries beginning with '"'. This avoids having to type out the entire list of keywords THREE times (in the grammar, in a set of %token statements, and in a table of keywords accessed by yylex()) -- having three lists that must be consistent is a major maintenance headache. My keywords do not associate and have no semantic values.

There are two aspects to this bug:
   A) the documentation is incomplete, tantamount to being wrong.
   B) in the output file.tab.c, the array yytoknum[] is inside a
      "#ifdef YYPRINT".
I have not built bison 2.4.3, but the bison.info in its source tarball contains the incomplete documentation, and yytoknum[] appears to be still inside the offending #ifdef.

In both bison-2.4.3/doc/bison.info and on page 84 of http://www.gnu.org/software/bison/manual/bison.pdf , the example code to map a string terminal to the return value from yylex() is incomplete -- it only gives a loop over yytname[], without telling the user what value to return. The loop variable is i, and the value that must be returned is yytoknum[i]. But yytoknum[] is inside that #ifdef and is not available. An alternative would be to search yytranslate[] for the entry equal to i, and return its index; this is ugly at best.

IMHO the documentation should be updated, and that #ifdef surrounding yytoknum[] should be removed. I have not tried this in languages other than C.

The attached sxfread.y contains two statements preceded by /*BUG ...*/ that illustrate my workaround. This is simple code to illustrate the bug.

Tom Roberts
/*      sxfread.y */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
int yylex(void);
void yyerror(const char *);
/*BUG This workaround makes yytoknum[] accesible. */
#define YYPRINT(A,B,C) /* need to enable yytoknum[] */
input:  /* EMPTY */
        | input line
line:   "drift" '{' description '}'
        | "quadrupole" '{' description '}'
description:    /* EMPTY */
        | description word
word:   'w'
int yylex(void)
        static char * token[] = {
        static int index=0;
        if(index >= sizeof(token)/sizeof(token[0])) {
                return 0;
        char *t=token[index++];
        if(t[1] == '\0') {
printf("returning %d\n",t[0]);
                return t[0]; /* single-char tokens are themselves */
        { char seek[259]; int i; /* this is C, not C++ */
          seek[0] = '"';
          for(i=0; i<YYNTOKENS; ++i) {
                if(strcmp(seek,yytname[i]) == 0) {
printf("returning %d\n",yytoknum[i]);
/*BUG This is the value that must be returned: */
                        return yytoknum[i];
printf("returning %d\n",2);
        return 2; /* error */

void yyerror (const char *s) 
        fprintf (stderr, "%s\n", s); 

int main (void) 
        int i;
        for(i=0; i<YYNTOKENS; ++i) {
                printf("%d: %s\n",i,yytname[i]);
        return yyparse (); 

reply via email to

[Prev in Thread] Current Thread [Next in Thread]