[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Enum for token '0', EOF
From: |
Joel E. Denny |
Subject: |
Re: Enum for token '0', EOF |
Date: |
Sat, 29 Jul 2006 00:13:29 -0400 (EDT) |
On Tue, 25 Jul 2006, Joel E. Denny wrote:
> On Wed, 12 Jul 2006, Joel E. Denny wrote:
>
> > I went ahead. As far as I can tell, all the skeletons now (after my
> > uncommitted patch below) consistently pop EOF one time upon a successful
> > parse.
>
> I'd like to commit this patch. I'll do so in a few days if there are no
> objections.
> The full patch is here:
>
> http://lists.gnu.org/archive/html/help-bison/2006-07/msg00023.html
I committed the following.
Joel
Index: ChangeLog
===================================================================
RCS file: /sources/bison/bison/ChangeLog,v
retrieving revision 1.1538
diff -p -u -r1.1538 ChangeLog
--- ChangeLog 25 Jul 2006 06:45:05 -0000 1.1538
+++ ChangeLog 29 Jul 2006 04:10:32 -0000
@@ -1,3 +1,32 @@
+2006-07-29 Joel E. Denny <address@hidden>
+
+ Clean up handling of %destructor for the end token (token 0).
+ Discussed starting at
+ <http://lists.gnu.org/archive/html/bison-patches/2006-07/msg00019.html>
+ and
+ <http://lists.gnu.org/archive/html/help-bison/2006-07/msg00013.html>.
+
+ Make the skeletons consistent in how they pop the end token and invoke
+ its %destructor.
+ * data/glr.c (yyrecoverSyntaxError, yyparse): Don't pop the start
+ state, which has token number 0, since this would invoke the
+ %destructor for the end token.
+ * data/lalr1.cc (yy::parser::parse): Don't check for the final state
+ until after shifting the end token, or else it won't be popped.
+ * data/yacc.c (yyparse): Likewise.
+
+ * data/glr.c (yyparse): Clear the lookahead after shifting it even when
+ it's the end token. Upon termination, destroy an unshifted lookahead
+ even when it's the end token.
+ * data/lalr1.cc (yy::parser::parse): Likewise.
+ * data/yacc.c (yyparse): Likewise.
+
+ * src/reader.c (packgram): Don't check rule 0. This suppresses unused
+ value warnings for the end token when the user gives the end token a
+ %destructor.
+
+ * tests/actions.at (Printers and Destructors): Test all the above.
+
2006-07-24 Paul Eggert <address@hidden>
Update to latest gnulib and gettext versions.
Index: data/glr.c
===================================================================
RCS file: /sources/bison/bison/data/glr.c,v
retrieving revision 1.185
diff -p -u -r1.185 glr.c
--- data/glr.c 9 Jul 2006 20:36:33 -0000 1.185
+++ data/glr.c 29 Jul 2006 04:10:33 -0000
@@ -2261,7 +2261,8 @@ yyrecoverSyntaxError (yyGLRStack* yystac
}
}
]b4_locations_if([[ yystackp->yyerror_range[1].yystate.yyloc =
yys->yyloc;]])[
- yydestroyGLRState ("Error: popping", yys]b4_user_args[);
+ if (yys->yypred != NULL)
+ yydestroyGLRState ("Error: popping", yys]b4_user_args[);
yystackp->yytops.yystates[0] = yys->yypred;
yystackp->yynextFree -= 1;
yystackp->yyspaceLeft += 1;
@@ -2373,8 +2374,7 @@ m4_popdef([b4_at_dollar])])dnl
if (yyisShiftAction (yyaction))
{
YY_SYMBOL_PRINT ("Shifting", yytoken, &yylval, &yylloc);
- if (yychar != YYEOF)
- yychar = YYEMPTY;
+ yychar = YYEMPTY;
yyposn += 1;
yyglrShift (&yystack, 0, yyaction, yyposn, &yylval, &yylloc);
if (0 < yystack.yyerrState)
@@ -2490,7 +2490,7 @@ m4_popdef([b4_at_dollar])])dnl
goto yyreturn;
yyreturn:
- if (yychar != YYEOF && yychar != YYEMPTY)
+ if (yychar != YYEMPTY)
yydestruct ("Cleanup: discarding lookahead",
YYTRANSLATE (yychar),
&yylval]b4_locations_if([, &yylloc])[]b4_user_args[);
@@ -2512,7 +2512,8 @@ m4_popdef([b4_at_dollar])])dnl
{
yyGLRState *yys = yystates[yyk];
]b4_locations_if([[ yystack.yyerror_range[1].yystate.yyloc =
yys->yyloc;]]
-)[ yydestroyGLRState ("Cleanup: popping", yys]b4_user_args[);
+)[ if (yys->yypred != NULL)
+ yydestroyGLRState ("Cleanup: popping", yys]b4_user_args[);
yystates[yyk] = yys->yypred;
yystack.yynextFree -= 1;
yystack.yyspaceLeft += 1;
Index: data/lalr1.cc
===================================================================
RCS file: /sources/bison/bison/data/lalr1.cc,v
retrieving revision 1.138
diff -p -u -r1.138 lalr1.cc
--- data/lalr1.cc 12 Jul 2006 18:01:19 -0000 1.138
+++ data/lalr1.cc 29 Jul 2006 04:10:33 -0000
@@ -567,6 +567,11 @@ m4_popdef([b4_at_dollar])])dnl
yynewstate:
yystate_stack_.push (yystate);
YYCDEBUG << "Entering state " << yystate << std::endl;
+
+ /* Accept? */
+ if (yystate == yyfinal_)
+ goto yyacceptlab;
+
goto yybackup;
/* Backup. */
@@ -616,16 +621,11 @@ m4_ifdef([b4_lex_param], [, ]b4_lex_para
goto yyreduce;
}
- /* Accept? */
- if (yyn == yyfinal_)
- goto yyacceptlab;
-
/* Shift the lookahead token. */
YY_SYMBOL_PRINT ("Shifting", yytoken, &yylval, &yylloc);
- /* Discard the token being shifted unless it is eof. */
- if (yychar != yyeof_)
- yychar = yyempty_;
+ /* Discard the token being shifted. */
+ yychar = yyempty_;
yysemantic_stack_.push (yylval);
yylocation_stack_.push (yylloc);
@@ -780,9 +780,6 @@ b4_error_verbose_if([, yytoken])[));
YY_STACK_PRINT ();
}
- if (yyn == yyfinal_)
- goto yyacceptlab;
-
yyerror_range[1] = yylloc;
// Using YYLLOC is tempting, but would change the location of
// the lookahead. YYLOC is available though.
@@ -808,7 +805,7 @@ b4_error_verbose_if([, yytoken])[));
goto yyreturn;
yyreturn:
- if (yychar != yyeof_ && yychar != yyempty_)
+ if (yychar != yyempty_)
yydestruct_ ("Cleanup: discarding lookahead", yytoken, &yylval, &yylloc);
/* Do not reclaim the symbols of the rule which action triggered
Index: data/yacc.c
===================================================================
RCS file: /sources/bison/bison/data/yacc.c,v
retrieving revision 1.150
diff -p -u -r1.150 yacc.c
--- data/yacc.c 9 Jul 2006 20:36:33 -0000 1.150
+++ data/yacc.c 29 Jul 2006 04:10:34 -0000
@@ -1164,6 +1164,9 @@ m4_ifdef([b4_at_dollar_used], [[ yylsp[
YYDPRINTF ((stderr, "Entering state %d\n", yystate));
+ if (yystate == YYFINAL)
+ YYACCEPT;
+
goto yybackup;
/*-----------.
@@ -1213,9 +1216,6 @@ yybackup:
goto yyreduce;
}
- if (yyn == YYFINAL)
- YYACCEPT;
-
/* Count tokens shifted since error; after three, turn off error
status. */
if (yyerrstatus)
@@ -1224,9 +1224,8 @@ yybackup:
/* Shift the lookahead token. */
YY_SYMBOL_PRINT ("Shifting", yytoken, &yylval, &yylloc);
- /* Discard the shifted token unless it is eof. */
- if (yychar != YYEOF)
- yychar = YYEMPTY;
+ /* Discard the shifted token. */
+ yychar = YYEMPTY;
yystate = yyn;
*++yyvsp = yylval;
@@ -1418,9 +1417,6 @@ yyerrlab1:
YY_STACK_PRINT (yyss, yyssp);
}
- if (yyn == YYFINAL)
- YYACCEPT;
-
*++yyvsp = yylval;
]b4_locations_if([[
yyerror_range[1] = yylloc;
@@ -1461,7 +1457,7 @@ yyexhaustedlab:
#endif
yyreturn:
- if (yychar != YYEOF && yychar != YYEMPTY)
+ if (yychar != YYEMPTY)
yydestruct ("Cleanup: discarding lookahead",
yytoken, &yylval]b4_locations_if([, &yylloc])[]b4_user_args[);
/* Do not reclaim the symbols of the rule which action triggered
Index: src/reader.c
===================================================================
RCS file: /sources/bison/bison/src/reader.c,v
retrieving revision 1.265
diff -p -u -r1.265 reader.c
--- src/reader.c 9 Jul 2006 20:36:33 -0000 1.265
+++ src/reader.c 29 Jul 2006 04:10:34 -0000
@@ -488,7 +488,11 @@ packgram (void)
rule. Thus, the midrule actions have already been scanned in order to
set `used' flags for this rule's rhs, so grammar_rule_check will work
properly. */
- grammar_rule_check (p);
+ /* Don't check the generated rule 0. It has no action, so some rhs
+ symbols may appear unused, but the parsing algorithm ensures that
+ %destructor's are invoked appropriately. */
+ if (p != grammar)
+ grammar_rule_check (p);
for (p = p->next; p && p->sym; p = p->next)
{
Index: tests/actions.at
===================================================================
RCS file: /sources/bison/bison/tests/actions.at,v
retrieving revision 1.60
diff -p -u -r1.60 actions.at
--- tests/actions.at 23 Jun 2006 20:17:28 -0000 1.60
+++ tests/actions.at 29 Jul 2006 04:10:34 -0000
@@ -197,8 +197,9 @@ AT_LALR1_CC_IF([typedef yy::location YYL
]AT_LALR1_CC_IF([], [static void yyerror (const char *msg);])
[}
-]m4_ifval([$6], [%type <ival> '(' 'x' 'y' ')' ';' thing line input])[
+]m4_ifval([$6], [%type <ival> '(' 'x' 'y' ')' ';' thing line input END])[
+/* FIXME: This %printer isn't actually tested. */
%printer
{
]AT_LALR1_CC_IF([debug_stream () << $$;],
@@ -226,6 +227,11 @@ AT_LALR1_CC_IF([typedef yy::location YYL
{ printf ("Freeing token 'y' (address@hidden)\n", $$, RANGE (@$)); }
'y'
+%token END 0
+%destructor
+ { printf ("Freeing token END (address@hidden)\n", $$, RANGE (@$)); }
+ END
+
%%
/*
This grammar is made to exercise error recovery.
@@ -241,7 +247,7 @@ input:
printf ("input (address@hidden): /* Nothing */\n", $$, RANGE (@$));
}
| line input /* Right recursive to load the stack so that popping at
- EOF can be exercised. */
+ END can be exercised. */
{
$$ = 2;
printf ("input (address@hidden): line (address@hidden) input
(address@hidden)\n",
@@ -308,7 +314,7 @@ yylex (]AT_LEX_FORMALS[)
if (source[c])
printf ("sending: '%c'", source[c]);
else
- printf ("sending: EOF");
+ printf ("sending: END");
printf (" (address@hidden)\n", c, RANGE (]AT_LOC[));
return source[c];
}
@@ -372,9 +378,10 @@ sending: 'x' (address@hidden)
thing (address@hidden): 'x' (address@hidden)
sending: ')' (address@hidden)
line (address@hidden): '(' (address@hidden) thing (address@hidden) ')'
(address@hidden)
-sending: EOF (address@hidden)
+sending: END (address@hidden)
input (address@hidden): /* Nothing */
input (address@hidden): line (address@hidden) input (address@hidden)
+Freeing token END (address@hidden)
Freeing nterm input (address@hidden)
Successful parse.
]])
@@ -391,9 +398,10 @@ sending: 'y' (address@hidden)
Freeing token 'y' (address@hidden)
sending: ')' (address@hidden)
line (address@hidden): '(' (address@hidden) error (@10-19) ')' (address@hidden)
-sending: EOF (address@hidden)
+sending: END (address@hidden)
input (address@hidden): /* Nothing */
input (address@hidden): line (address@hidden) input (address@hidden)
+Freeing token END (address@hidden)
Freeing nterm input (address@hidden)
Successful parse.
]])
@@ -445,12 +453,45 @@ input (address@hidden): /* Nothing */
input (address@hidden): line (address@hidden) input (address@hidden)
input (address@hidden): line (address@hidden) input (address@hidden)
input (address@hidden): line (address@hidden) input (address@hidden)
-130-139: syntax error, unexpected 'y', expecting $end
+130-139: syntax error, unexpected 'y', expecting END
Freeing nterm input (address@hidden)
Freeing token 'y' (address@hidden)
Parsing FAILED.
]])
+
+# Syntax error caught by the parser where lookahead = END
+# --------------------------------------------------------
+# Load the stack and provoke an error that cannot be caught by the
+# grammar, to check that the stack is cleared. And make sure the
+# lookahead is freed.
+#
+# '(', 'x', ')',
+# '(', 'x', ')',
+# 'x'
+AT_PARSER_CHECK([./input '(x)(x)x'], 1,
+[[sending: '(' (address@hidden)
+sending: 'x' (address@hidden)
+thing (address@hidden): 'x' (address@hidden)
+sending: ')' (address@hidden)
+line (address@hidden): '(' (address@hidden) thing (address@hidden) ')'
(address@hidden)
+sending: '(' (address@hidden)
+sending: 'x' (address@hidden)
+thing (address@hidden): 'x' (address@hidden)
+sending: ')' (address@hidden)
+line (address@hidden): '(' (address@hidden) thing (address@hidden) ')'
(address@hidden)
+sending: 'x' (address@hidden)
+thing (address@hidden): 'x' (address@hidden)
+sending: END (address@hidden)
+70-79: syntax error, unexpected END, expecting 'x'
+Freeing nterm thing (address@hidden)
+Freeing nterm line (address@hidden)
+Freeing nterm line (address@hidden)
+Freeing token END (address@hidden)
+Parsing FAILED.
+]])
+
+
# Check destruction upon stack overflow
# -------------------------------------
# Upon stack overflow, all symbols on the stack should be destroyed.
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: Enum for token '0', EOF,
Joel E. Denny <=