Re: `YYERROR;' should not discard lookahead token

[Akim Demaille]

| POSIX 1003.1-2001 says that a Yacc-generated parser should not discard
| the lookahead token merely because a user action invokes `YYERROR;',
| so I installed this patch.  (This patch also removes an obsolete,
| unrelated comment.)
| 
| 2002-11-24  Paul Eggert  <address@hidden>
| 
|       * data/yacc.c (yyerrlab1): Move this label down, so that the
|       parser does not discard the lookahead token if the user code
|       invokes YYERROR.  This change is required for POSIX conformance.
| 
| Index: yacc.c
| ===================================================================
| RCS file: /cvsroot/bison/bison/data/yacc.c,v
| retrieving revision 1.38
| retrieving revision 1.39
| diff -p -u -r1.38 -r1.39
| --- yacc.c    16 Nov 2002 12:32:27 -0000      1.38
| +++ yacc.c    25 Nov 2002 06:42:10 -0000      1.39
| @@ -912,8 +912,6 @@ yybackup:
|      }
|    else
|      {
| -      /* We have to keep this `#if YYDEBUG', since we use variables
| -      which are defined only if `YYDEBUG' is set.  */
|        YYDSYMPRINTF ("Next token is", yytoken, &yylval, &yylloc);
|      }
|  
| @@ -1073,13 +1071,7 @@ yyerrlab:
|  #endif /* YYERROR_VERBOSE */
|       yyerror (]b4_yyerror_args["syntax error");
|      }
| -  goto yyerrlab1;
| -
|  
| -/*----------------------------------------------------.
| -| yyerrlab1 -- error raised explicitly by an action.  |
| -`----------------------------------------------------*/
| -yyerrlab1:
|    if (yyerrstatus == 3)
|      {
|        /* If just tried and failed to reuse lookahead token after an
| @@ -1107,7 +1099,13 @@ yyerrlab1:
|  
|    /* Else will try to reuse lookahead token after shifting the error
|       token.  */
| +  goto yyerrlab1;
| +
|  
| +/*----------------------------------------------------.
| +| yyerrlab1 -- error raised explicitly by an action.  |
| +`----------------------------------------------------*/
| +yyerrlab1:
|    yyerrstatus = 3;   /* Each real token shifted decrements this.  */
|  
|    for (;;)

Is there any test related to this feature?  Also, I don't absolutely
understand how the patch and the ChangeLog relate to each other,
since, if I understand correctly, what you actually changed is the
behavior of YYERROR when we are already in error mode.  (I'm reading
the code as it is now, so maybe the context is no longer the same:

/*------------------------------------.
| yyerrlab -- here on detecting error |
`------------------------------------*/
yyerrlab:
  /* If not already recovering from an error, report this error.  */
  if (!yyerrstatus)
    {
      ++yynerrs;
#if YYERROR_VERBOSE
[...]
#endif /* YYERROR_VERBOSE */
        yyerror (]b4_yyerror_args["syntax error");
    }

]b4_location_if([  yylerrsp = yylsp;])[

  if (yyerrstatus == 3)
    {
      /* If just tried and failed to reuse lookahead token after an
         error, discard it.  */
    [...]
    }

  /* Else will try to reuse lookahead token after shifting the error
     token.  */
  goto yyerrlab1;


/*----------------------------------------------------.
| yyerrlab1 -- error raised explicitly by an action.  |
`----------------------------------------------------*/
yyerrlab1:
  yyerrstatus = 3;      /* Each real token shifted decrements this.  */