Re: push parser

[Bob Rossi]

Hi Paul,

This is the "larger patch" I was hoping you would apply. After this,
I'll start sending in small patches to fine tune push.c. That should
help us get to our goal of copying push.c over to yacc.c

Thanks,
Bob Rossi

On Wed, Oct 11, 2006 at 04:48:57PM -0400, Bob Rossi wrote:
> Hi,
> 
> Paul, are you going to apply the patch to push.c? I think it does
> pass the testsuite if you 'cp push.c yacc.c'. This would be helpful
> to me if you could. Then I could give one final annotation of
> yacc and push, before we copy them over.
> 
> Thanks,
> Bob Rossi
> 
> On Mon, Oct 02, 2006 at 10:24:36AM -0400, Bob Rossi wrote:
> > On Fri, Sep 29, 2006 at 11:30:51PM -0400, Bob Rossi wrote:
> > > On Fri, Sep 29, 2006 at 05:55:08PM -0700, Paul Eggert wrote:
> > > > Bob Rossi <address@hidden> writes:
> > > > 
> > > > > If we force the user to define that function, it add's an unnecessary 
> > > > > complication to the push parser. That is, the user is mostly likely 
> > > > > going to call yypushparse, and not care about yyparse at all. However,
> > > > > yyparse is the function causing the user to define the yypushlex
> > > > > function. What do you think I should do to resolve this?
> > > > 
> > > > Ah, sorry, I didn't understand this issue at all.
> > > > 
> > > > It sounds to me like the push parser shouldn't define yyparse.  That
> > > > way, the user shouldn't have to care about any lexer function.
> > > 
> > > I agree. Akim, you were the main advocate for adding this functionality. 
> > > Can
> > > you see any other solution to the problem besides remove yyparse from
> > > the generated output?
> > 
> > Akim, when you get around to this, let me know what your thoughts are.
> > For now, I've removed yyparse again in push mode. It can be easily added
> > back ...
> > 
> > Paul, I think we are getting closer to a userful interface for the user.
> > Below is what it currently looks like.
> > 
> >   struct yypvars *ctx = yypvarsinit ();
> >   int status;
> >   YYSTYPE my_lval;
> >   YYLTYPE my_lloc;
> >   do {
> >     status = yypushparse (ctx, yylex (&my_lval, &my_lloc), &my_lval, 
> > &my_lloc)
> >   } while (status == YYPUSH_MORE);
> >   free (ctx);
> > 
> > The user is forced to pass in NULL to my_lval. When locations are turned
> > on the extra parameter will also have to be passed in. Otherwise, not.
> > 
> > How is this looking? I'm starting to like it much better than what was 
> > previously done.
> > 
> > Thanks,
> > Bob Rossi
> > 
> > 2006-10-02  Bob Rossi  <address@hidden>
> > 
> >         * data/push.c (yychar_set, yylval_set, yylloc_set): Delete.
> >         (yypushparse): Add yynchar, yynlval, yynlloc parameters.
> >         (b4_declare_parser_variables): Do not declare yynerrs for push mode.
> >         (struct yypvars): Remove b4_declare_parser_variables.
> >         (yypvarsinit): Remove init code for removed variables.
> >         (global scope): Do not declare b4_declare_parser_variables if
> >         push or pure mode.
> >         (yypushparse): Add b4_declare_parser_variables.
> >         Init new local variables, and remove init code for removed
> >         yypvars variables.
> >         (yyparse): Delete.
> >         * tests/calc.at (_AT_DATA_CALC_Y): Call yypushparse for push mode
> >         and yyparse for other modes.
> >         * tests/local.at (AT_PUSH_IF): Added.
> >         (AT_PURE_IF): Rename to AT_PURE_OR_PUSH_IF and modify accordingly.
> >         (AT_PURE_AND_LOC_IF): Rename to AT_PURE_OR_PUSH_AND_LOC_IF and
> >         modify accordingly.
> >         (AT_YYERROR_ARG_LOC_IF): Use AT_PURE_OR_PUSH_AND_LOC_IF.
> >         (AT_YYERROR_SEES_LOC_IF): Use AT_PURE_OR_PUSH_IF.
> >         (AT_PURE_LEX_IF): Use AT_PURE_OR_PUSH_IF.
> > 
> > 
> 
> > Index: data/push.c
> > ===================================================================
> > RCS file: /sources/bison/bison/data/push.c,v
> > retrieving revision 1.3
> > diff -u -r1.3 push.c
> > --- data/push.c     21 Sep 2006 17:45:21 -0000      1.3
> > +++ data/push.c     2 Oct 2006 14:12:27 -0000
> > @@ -161,11 +161,7 @@
> >  #define yychar  b4_prefix[]char
> >  #define yydebug b4_prefix[]debug
> >  #define yynerrs b4_prefix[]nerrs
> > -b4_locations_if([#define yylloc b4_prefix[]lloc])
> > -b4_push_if([
> > -#define yychar_set b4_prefix[]char_set
> > -#define yylval_set b4_prefix[]lval_set
> > -#define yylloc_set b4_prefix[]lloc_set])])[
> > +b4_locations_if([#define yylloc b4_prefix[]lloc])])[
> >  
> >  /* Copy the first part of user declarations.  */
> >  ]b4_pre_prologue[
> > @@ -974,14 +970,12 @@
> >  ]b4_push_if([
> >  struct yypvars;
> >  enum { YYPUSH_MORE = 4 };
> > -]b4_c_function_decl([yychar_set], [void], [[struct yypvars *YYPVARS], 
> > [YYPVARS]], [[int yychar], [yychar]])[
> > -]b4_c_function_decl([yylval_set], [void], [[struct yypvars *YYPVARS], 
> > [YYPVARS]], [[YYSTYPE yylval], [yylval]])[
> > -#ifdef YYLTYPE_IS_TRIVIAL
> > -]b4_c_function_decl([yylloc_set], [void], [[struct yypvars *YYPVARS], 
> > [YYPVARS]], [[YYLTYPE yylloc], [yylloc]])[
> > -#endif
> >  ]b4_c_function_decl([yypvarsinit], [void *], [[void], []])[
> >  ]b4_c_function_decl([yypushparse], [int],
> > -   [[struct yypvars *YYPVARS], [YYPVARS]])[
> > +   [[struct yypvars *yypvars], [yypvars]],
> > +   [[int yynchar], [yynchar]],
> > +   [[YYSTYPE *yynlval], [yynlval]]
> > +   b4_locations_if([,[[YYLTYPE *yynlloc], [yynlloc]]]))[
> >  ])[
> >  
> >  ]m4_divert_push([KILL])# ======================== M4 code.
> > @@ -997,7 +991,8 @@
> >  YYSTYPE yylval;
> >  
> >  /* Number of syntax errors so far.  */
> > -int yynerrs;b4_locations_if([
> > +]b4_push_if([],[
> > +int yynerrs;])[b4_locations_if([
> >  /* Location data for the lookahead symbol.  */
> >  YYLTYPE yylloc;])
> >  ])
> > @@ -1008,7 +1003,6 @@
> >  m4_define([b4_declare_yyparse_variables],
> >  [[struct yypvars
> >    {
> > -]]b4_declare_parser_variables[[
> >      int yystate;
> >      int yyn;
> >      int yyresult;
> > @@ -1087,45 +1081,16 @@
> >    pv->yyssp = pv->yyss;
> >    pv->yyvsp = pv->yyvs;
> >  
> > -#if YYLTYPE_IS_TRIVIAL
> > -  /* Initialize the default location before parsing starts.  */
> > -  pv->yylloc.first_line   = pv->yylloc.last_line   = 
> > ]b4_location_initial_line[;
> > -  pv->yylloc.first_column = pv->yylloc.last_column = 
> > ]b4_location_initial_column[;
> > -#endif
> > -
> >    pv->yynew = 1;
> >  
> >  ]b4_locations_if([  pv->yylsp = pv->yyls;])[
> >  
> >    return (void *) pv;
> > -}
> > -
> > -void
> > -yychar_set (struct yypvars *YYPVARS, int yychar)
> > -{
> > -  if (YYPVARS)
> > -    YYPVARS->yychar = yychar;
> > -}
> > -
> > -void
> > -yylval_set (struct yypvars *YYPVARS, YYSTYPE yylval)
> > -{
> > -  if (YYPVARS)
> > -    YYPVARS->yylval = yylval;
> > -}
> > -
> > -#ifdef YYLTYPE_IS_TRIVIAL
> > -void
> > -yylloc_set (struct yypvars *YYPVARS, YYLTYPE yylloc)
> > -{
> > -  if (YYPVARS)
> > -    YYPVARS->yylloc = yylloc;
> > -}
> > -#endif])
> > +}])
> >  m4_divert_pop([KILL])dnl# ====================== End of M4 code.
> >  
> > -b4_pure_if([],
> > -      [b4_declare_parser_variables])
> > +b4_push_if([],[b4_pure_if([],
> > +      [b4_declare_parser_variables])])
> >  
> >  b4_push_if([b4_declare_yyparse_variables])
> >  
> > @@ -1134,7 +1099,9 @@
> >  `-------------------------*/
> >  
> >  b4_push_if([
> > -b4_c_function_def([yypushparse], [int], [[struct yypvars *YYPVARS], 
> > [YYPVARS]])],[
> > +b4_c_function_def([yypushparse], [int], [[struct yypvars *yypvars], 
> > [yypvars]], 
> > +                  [[int yynchar], [yynchar]], [[YYSTYPE *yynlval], 
> > [yynlval]]
> > +             b4_locations_if([,[[YYLTYPE *yynlloc], [yynlloc]]]))],[
> >  #ifdef YYPARSE_PARAM
> >  b4_c_function_def([yyparse], [int], [[void *YYPARSE_PARAM], 
> > [YYPARSE_PARAM]])
> >  #else /* ! YYPARSE_PARAM */
> > @@ -1142,6 +1109,7 @@
> >  #endif])
> >  {[
> >    ]b4_pure_if([b4_declare_parser_variables])[
> > +  ]b4_push_if([b4_declare_parser_variables])[
> >    ]b4_push_if([struct yypvars *pv;])[
> >    int yystate;
> >    int yyn;
> > @@ -1202,12 +1170,15 @@
> >  
> >    YYDPRINTF ((stderr, "Starting parse\n"));
> >  
> > -  ]b4_push_if([pv = YYPVARS;])[
> > -
> >    yystate = 0;
> >    yyerrstatus = 0;
> > -  yynerrs = 0;
> > -  yychar = YYEMPTY;                /* Cause a token to be read.  */
> > +]b4_push_if([  yychar = yynchar;
> > +  pv = yypvars;
> > +  if (yynlval)
> > +    yylval = *yynlval;
> > +]b4_locations_if([  if (yynlloc)
> > +     yylloc = *yynlloc;])[],[yynerrs = 0;
> > +  yychar = YYEMPTY; /* Cause a token to be read.  */])[
> >  
> >    /* Initialize stack pointers.
> >       Waste one element of value and location stack
> > @@ -1237,12 +1208,6 @@
> >  ]])dnl
> >  [  ]b4_push_if([
> >     /* Initialize the locals to the current context. */
> > -   yychar = pv->yychar;
> > -   yylval = pv->yylval;
> > -   yynerrs = pv->yynerrs;
> > -   ]b4_locations_if([
> > -   yylloc = pv->yylloc;])[
> > -
> >     yystate = pv->yystate;
> >     yyn = pv->yyn;
> >     yyresult = pv->yyresult;
> > @@ -1382,12 +1347,6 @@
> >     YYDPRINTF ((stderr, "Return for a new token:\n"));
> >     yyresult = YYPUSH_MORE;
> >     /* Initialize the locals to the current context. */
> > -   pv->yychar = yychar;
> > -   pv->yylval = yylval;
> > -   pv->yynerrs = yynerrs;
> > -   ]b4_locations_if([
> > -   pv->yylloc = yylloc;])[
> > -
> >     pv->yystate = yystate;
> >     pv->yyn = yyn;
> >     pv->yyresult = yyresult;
> > @@ -1530,7 +1489,7 @@
> >    /* If not already recovering from an error, report this error.  */
> >    if (!yyerrstatus)
> >      {
> > -      ++yynerrs;
> > +]b4_push_if([],[ ++yynerrs;])[
> >  #if ! YYERROR_VERBOSE
> >        yyerror (]b4_yyerror_args[YY_("syntax error"));
> >  #else
> > @@ -1713,28 +1672,6 @@
> >      return YYID (yyresult);
> >  ]}
> >  
> > -b4_push_if([
> > -#ifdef YYPARSE_PARAM
> > -b4_c_function_def([yyparse], [int], [[void *YYPARSE_PARAM], 
> > [YYPARSE_PARAM]])
> > -#else /* ! YYPARSE_PARAM */
> > -b4_c_function_def([yyparse], [int], b4_parse_param)
> > -#endif
> > -{[
> > -  struct yypvars *ctx = yypvarsinit ();
> > -  int status;
> > -  do {
> > -    yychar_set (ctx, yylex ());
> > -    yylval_set (ctx, yylval);
> > -#ifdef YYLTYPE_IS_TRIVIAL
> > -    yylloc_set (ctx, yylloc);
> > -#endif
> > -    status = yypushparse (ctx);
> > -  } while (status == YYPUSH_MORE);
> > -  free (ctx);
> > -  return status;
> > -]}])
> > -
> > -
> >  b4_epilogue
> >  b4_defines_if(
> >  address@hidden @output_header_name@
> > Index: tests/calc.at
> > ===================================================================
> > RCS file: /sources/bison/bison/tests/calc.at,v
> > retrieving revision 1.92
> > diff -u -r1.92 calc.at
> > --- tests/calc.at   15 Sep 2006 15:56:26 -0000      1.92
> > +++ tests/calc.at   2 Oct 2006 14:12:27 -0000
> > @@ -336,7 +336,17 @@
> >  
> >  ]AT_SKEL_CC_IF([], [m4_bmatch([$4], [%debug],
> >  [  yydebug = 1;])])[
> > -  status = yyparse (]AT_PARAM_IF([&result, &count])[);
> > +]AT_PUSH_IF([
> > +  {
> > +    struct yypvars *ctx = yypvarsinit ();
> > +    YYSTYPE my_lval;
> > +    ]AT_LOCATION_IF([YYLTYPE my_lloc;])[
> > +    do {
> > +      status = yypushparse (ctx, yylex (&my_lval]AT_LOCATION_IF([[, 
> > &my_lloc]])[), &my_lval]AT_LOCATION_IF([[, &my_lloc]])[);
> > +    } while (status == YYPUSH_MORE);
> > +    free (ctx);
> > +  }],[
> > +  status = yyparse (]AT_PARAM_IF([[&result, &count]])[);])[
> >    if (global_result != result)
> >      abort ();
> >    if (global_count != count)
> > Index: tests/local.at
> > ===================================================================
> > RCS file: /sources/bison/bison/tests/local.at,v
> > retrieving revision 1.16
> > diff -u -r1.16 local.at
> > --- tests/local.at  12 Mar 2006 15:26:05 -0000      1.16
> > +++ tests/local.at  2 Oct 2006 14:12:28 -0000
> > @@ -56,10 +56,12 @@
> >  [m4_bmatch([$3], [%parse-param], [$1], [$2])])
> >  m4_pushdef([AT_LOCATION_IF],
> >  [m4_bmatch([$3], [%locations], [$1], [$2])])
> > -m4_pushdef([AT_PURE_IF],
> > -[m4_bmatch([$3], [%pure-parser], [$1], [$2])])
> > -m4_pushdef([AT_PURE_AND_LOC_IF],
> > -[m4_bmatch([$3], [%locations.*%pure-parser\|%pure-parser.*%locations],
> > +m4_pushdef([AT_PUSH_IF],
> > +[m4_bmatch([$3], [%push-parser], [$1], [$2])])
> > +m4_pushdef([AT_PURE_OR_PUSH_IF],
> > +[m4_bmatch([$3], [%pure-parser\|%push-parser], [$1], [$2])])
> > +m4_pushdef([AT_PURE_OR_PUSH_AND_LOC_IF],
> > +[m4_bmatch([$3], 
> > [%locations.*%pure-parser\|%pure-parser.*%locations\|%locations.*%push-parser\|%push-parser.*%locations],
> >        [$1], [$2])])
> >  m4_pushdef([AT_GLR_OR_PARAM_IF],
> >  [m4_bmatch([$3], [%glr-parser\|%parse-param], [$1], [$2])])
> > @@ -69,12 +71,12 @@
> >             [yy])])
> >  # yyerror receives the location if %location & %pure & (%glr or 
> > %parse-param).
> >  m4_pushdef([AT_YYERROR_ARG_LOC_IF],
> > -[AT_GLR_OR_PARAM_IF([AT_PURE_AND_LOC_IF([$1], [$2])],
> > +[AT_GLR_OR_PARAM_IF([AT_PURE_OR_PUSH_AND_LOC_IF([$1], [$2])],
> >                 [$2])])
> >  # yyerror always sees the locations (when activated), except if
> >  # yacc & pure & !param.
> >  m4_pushdef([AT_YYERROR_SEES_LOC_IF],
> > -[AT_LOCATION_IF([AT_YACC_IF([AT_PURE_IF([AT_PARAM_IF([$1], [$2])],
> > +[AT_LOCATION_IF([AT_YACC_IF([AT_PURE_OR_PUSH_IF([AT_PARAM_IF([$1], [$2])],
> >                                     [$1])],
> >                         [$1])],
> >             [$2])])
> > @@ -82,7 +84,7 @@
> >  # The interface is pure: either because %pure-parser, or because we
> >  # are using the C++ parsers.
> >  m4_pushdef([AT_PURE_LEX_IF],
> > -[AT_PURE_IF([$1],
> > +[AT_PURE_OR_PUSH_IF([$1],
> >         [AT_SKEL_CC_IF([$1], [$2])])])
> >  
> >  AT_PURE_LEX_IF(
> > @@ -125,7 +127,7 @@
> >  m4_popdef([AT_YYERROR_ARG_LOC_IF])
> >  m4_popdef([AT_NAME_PREFIX])
> >  m4_popdef([AT_GLR_OR_PARAM_IF])
> > -m4_popdef([AT_PURE_AND_LOC_IF])
> > +m4_popdef([AT_PURE_OR_PUSH_AND_LOC_IF])
> >  m4_popdef([AT_LOCATION_IF])
> >  m4_popdef([AT_PARAM_IF])
> >  m4_popdef([AT_YACC_IF])
> 
> 
>