bison-3.0 released [stable]

[Akim Demaille]

The Bison team is very happy to announce the release of Bison 3.0, which
introduces many new features.  An executive summary would include: (i) deep
overhaul/improvements of the diagnostics, (ii) more versatile means to
describe semantic value types (including the ability to store genuine C++
objects in C++ parsers), (iii) push-parser interface extended to Java, and
(iv) parse-time semantic predicates for GLR parsers.

Here are the compressed sources:
  ftp://ftp.gnu.org/gnu/bison/bison-3.0.tar.gz   (3.1MB)
  ftp://ftp.gnu.org/gnu/bison/bison-3.0.tar.xz   (1.8MB)

Here are the GPG detached signatures[*]:
  ftp://ftp.gnu.org/gnu/bison/bison-3.0.tar.gz.sig
  ftp://ftp.gnu.org/gnu/bison/bison-3.0.tar.xz.sig

Use a mirror for higher download bandwidth:
  http://www.gnu.org/order/ftp.html

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify bison-3.0.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.14
  Flex 2.5.37
  Gettext 0.18.3
  Gnulib v0.0-7982-g03e96cc

NEWS

* Noteworthy changes in release 3.0 (2013-07-25) [stable]

** WARNING: Future backward-incompatibilities!

  Like other GNU packages, Bison will start using some of the C99 features
  for its own code, especially the definition of variables after statements.
  The generated C parsers still aim at C90.

** Backward incompatible changes

*** Obsolete features

  Support for YYFAIL is removed (deprecated in Bison 2.4.2): use YYERROR.

  Support for yystype and yyltype is removed (deprecated in Bison 1.875):
  use YYSTYPE and YYLTYPE.

  Support for YYLEX_PARAM and YYPARSE_PARAM is removed (deprecated in Bison
  1.875): use %lex-param, %parse-param, or %param.

  Missing semicolons at the end of actions are no longer added (as announced
  in the release 2.5).

*** Use of YACC='bison -y'

  TL;DR: With Autoconf <= 2.69, pass -Wno-yacc to (AM_)YFLAGS if you use
  Bison extensions.

  Traditional Yacc generates 'y.tab.c' whatever the name of the input file.
  Therefore Makefiles written for Yacc expect 'y.tab.c' (and possibly
  'y.tab.h' and 'y.outout') to be generated from 'foo.y'.

  To this end, for ages, AC_PROG_YACC, Autoconf's macro to look for an
  implementation of Yacc, was using Bison as 'bison -y'.  While it does
  ensure compatible output file names, it also enables warnings for
  incompatibilities with POSIX Yacc.  In other words, 'bison -y' triggers
  warnings for Bison extensions.

  Autoconf 2.70+ fixes this incompatibility by using YACC='bison -o y.tab.c'
  (which also generates 'y.tab.h' and 'y.output' when needed).
  Alternatively, disable Yacc warnings by passing '-Wno-yacc' to your Yacc
  flags (YFLAGS, or AM_YFLAGS with Automake).

** Bug fixes

*** The epilogue is no longer affected by internal #defines (glr.c)

  The glr.c skeleton uses defines such as #define yylval (yystackp->yyval) in
  generated code.  These weren't properly undefined before the inclusion of
  the user epilogue, so functions such as the following were butchered by the
  preprocessor expansion:

    int yylex (YYSTYPE *yylval);

  This is fixed: yylval, yynerrs, yychar, and yylloc are now valid
  identifiers for user-provided variables.

*** stdio.h is no longer needed when locations are enabled (yacc.c)

  Changes in Bison 2.7 introduced a dependency on FILE and fprintf when
  locations are enabled.  This is fixed.

*** Warnings about useless %pure-parser/%define api.pure are restored

** Diagnostics reported by Bison

  Most of these features were contributed by Théophile Ranquet and Victor
  Santet.

*** Carets

  Version 2.7 introduced caret errors, for a prettier output.  These are now
  activated by default.  The old format can still be used by invoking Bison
  with -fno-caret (or -fnone).

  Some error messages that reproduced excerpts of the grammar are now using
  the caret information only.  For instance on:

    %%
    exp: 'a' | 'a';

  Bison 2.7 reports:

    in.y: warning: 1 reduce/reduce conflict [-Wconflicts-rr]
    in.y:2.12-14: warning: rule useless in parser due to conflicts: exp: 'a' 
[-Wother]

  Now bison reports:

    in.y: warning: 1 reduce/reduce conflict [-Wconflicts-rr]
    in.y:2.12-14: warning: rule useless in parser due to conflicts [-Wother]
     exp: 'a' | 'a';
                ^^^

  and "bison -fno-caret" reports:

    in.y: warning: 1 reduce/reduce conflict [-Wconflicts-rr]
    in.y:2.12-14: warning: rule useless in parser due to conflicts [-Wother]

*** Enhancements of the -Werror option

  The -Werror=CATEGORY option is now recognized, and will treat specified
  warnings as errors. The warnings need not have been explicitly activated
  using the -W option, this is similar to what GCC 4.7 does.

  For example, given the following command line, Bison will treat both
  warnings related to POSIX Yacc incompatibilities and S/R conflicts as
  errors (and only those):

    $ bison -Werror=yacc,error=conflicts-sr input.y

  If no categories are specified, -Werror will make all active warnings into
  errors. For example, the following line does the same the previous example:

    $ bison -Werror -Wnone -Wyacc -Wconflicts-sr input.y

  (By default -Wconflicts-sr,conflicts-rr,deprecated,other is enabled.)

  Note that the categories in this -Werror option may not be prefixed with
  "no-". However, -Wno-error[=CATEGORY] is valid.

  Note that -y enables -Werror=yacc. Therefore it is now possible to require
  Yacc-like behavior (e.g., always generate y.tab.c), but to report
  incompatibilities as warnings: "-y -Wno-error=yacc".

*** The display of warnings is now richer

  The option that controls a given warning is now displayed:

    foo.y:4.6: warning: type clash on default action: <foo> != <bar> [-Wother]

  In the case of warnings treated as errors, the prefix is changed from
  "warning: " to "error: ", and the suffix is displayed, in a manner similar
  to GCC, as [-Werror=CATEGORY].

  For instance, where the previous version of Bison would report (and exit
  with failure):

    bison: warnings being treated as errors
    input.y:1.1: warning: stray ',' treated as white space

  it now reports:

    input.y:1.1: error: stray ',' treated as white space [-Werror=other]

*** Deprecated constructs

  The new 'deprecated' warning category flags obsolete constructs whose
  support will be discontinued.  It is enabled by default.  These warnings
  used to be reported as 'other' warnings.

*** Useless semantic types

  Bison now warns about useless (uninhabited) semantic types.  Since
  semantic types are not declared to Bison (they are defined in the opaque
  %union structure), it is %printer/%destructor directives about useless
  types that trigger the warning:

    %token <type1> term
    %type  <type2> nterm
    %printer    {} <type1> <type3>
    %destructor {} <type2> <type4>
    %%
    nterm: term { $$ = $1; };

    3.28-34: warning: type <type3> is used, but is not associated to any symbol
    4.28-34: warning: type <type4> is used, but is not associated to any symbol

*** Undefined but unused symbols

  Bison used to raise an error for undefined symbols that are not used in
  the grammar.  This is now only a warning.

    %printer    {} symbol1
    %destructor {} symbol2
    %type <type>   symbol3
    %%
    exp: "a";

*** Useless destructors or printers

  Bison now warns about useless destructors or printers.  In the following
  example, the printer for <type1>, and the destructor for <type2> are
  useless: all symbols of <type1> (token1) already have a printer, and all
  symbols of type <type2> (token2) already have a destructor.

    %token <type1> token1
           <type2> token2
           <type3> token3
           <type4> token4
    %printer    {} token1 <type1> <type3>
    %destructor {} token2 <type2> <type4>

*** Conflicts

  The warnings and error messages about shift/reduce and reduce/reduce
  conflicts have been normalized.  For instance on the following foo.y file:

    %glr-parser
    %%
    exp: exp '+' exp | '0' | '0';

  compare the previous version of bison:

    $ bison foo.y
    foo.y: conflicts: 1 shift/reduce, 2 reduce/reduce
    $ bison -Werror foo.y
    bison: warnings being treated as errors
    foo.y: conflicts: 1 shift/reduce, 2 reduce/reduce

  with the new behavior:

    $ bison foo.y
    foo.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
    foo.y: warning: 2 reduce/reduce conflicts [-Wconflicts-rr]
    $ bison -Werror foo.y
    foo.y: error: 1 shift/reduce conflict [-Werror=conflicts-sr]
    foo.y: error: 2 reduce/reduce conflicts [-Werror=conflicts-rr]

  When %expect or %expect-rr is used, such as with bar.y:

    %expect 0
    %glr-parser
    %%
    exp: exp '+' exp | '0' | '0';

  Former behavior:

    $ bison bar.y
    bar.y: conflicts: 1 shift/reduce, 2 reduce/reduce
    bar.y: expected 0 shift/reduce conflicts
    bar.y: expected 0 reduce/reduce conflicts

  New one:

    $ bison bar.y
    bar.y: error: shift/reduce conflicts: 1 found, 0 expected
    bar.y: error: reduce/reduce conflicts: 2 found, 0 expected

** Incompatibilities with POSIX Yacc

  The 'yacc' category is no longer part of '-Wall', enable it explicitly
  with '-Wyacc'.

** Additional yylex/yyparse arguments

  The new directive %param declares additional arguments to both yylex and
  yyparse.  The %lex-param, %parse-param, and %param directives support one
  or more arguments.  Instead of

    %lex-param   {arg1_type *arg1}
    %lex-param   {arg2_type *arg2}
    %parse-param {arg1_type *arg1}
    %parse-param {arg2_type *arg2}

  one may now declare

    %param {arg1_type *arg1} {arg2_type *arg2}

** Types of values for %define variables

  Bison used to make no difference between '%define foo bar' and '%define
  foo "bar"'.  The former is now called a 'keyword value', and the latter a
  'string value'.  A third kind was added: 'code values', such as '%define
  foo {bar}'.

  Keyword variables are used for fixed value sets, e.g.,

    %define lr.type lalr

  Code variables are used for value in the target language, e.g.,

    %define api.value.type {struct semantic_type}

  String variables are used remaining cases, e.g. file names.

** Variable api.token.prefix

  The variable api.token.prefix changes the way tokens are identified in
  the generated files.  This is especially useful to avoid collisions
  with identifiers in the target language.  For instance

    %token FILE for ERROR
    %define api.token.prefix {TOK_}
    %%
    start: FILE for ERROR;

  will generate the definition of the symbols TOK_FILE, TOK_for, and
  TOK_ERROR in the generated sources.  In particular, the scanner must
  use these prefixed token names, although the grammar itself still
  uses the short names (as in the sample rule given above).

** Variable api.value.type

  This new %define variable supersedes the #define macro YYSTYPE.  The use
  of YYSTYPE is discouraged.  In particular, #defining YYSTYPE *and* either
  using %union or %defining api.value.type results in undefined behavior.

  Either define api.value.type, or use "%union":

    %union
    {
      int ival;
      char *sval;
    }
    %token <ival> INT "integer"
    %token <sval> STRING "string"
    %printer { fprintf (yyo, "%d", $$); } <ival>
    %destructor { free ($$); } <sval>

    /* In yylex().  */
    yylval.ival = 42; return INT;
    yylval.sval = "42"; return STRING;

  The %define variable api.value.type supports both keyword and code values.

  The keyword value 'union' means that the user provides genuine types, not
  union member names such as "ival" and "sval" above (WARNING: will fail if
  -y/--yacc/%yacc is enabled).

    %define api.value.type union
    %token <int> INT "integer"
    %token <char *> STRING "string"
    %printer { fprintf (yyo, "%d", $$); } <int>
    %destructor { free ($$); } <char *>

    /* In yylex().  */
    yylval.INT = 42; return INT;
    yylval.STRING = "42"; return STRING;

  The keyword value variant is somewhat equivalent, but for C++ special
  provision is made to allow classes to be used (more about this below).

    %define api.value.type variant
    %token <int> INT "integer"
    %token <std::string> STRING "string"

  Code values (in braces) denote user defined types.  This is where YYSTYPE
  used to be used.

    %code requires
    {
      struct my_value
      {
        enum
        {
          is_int, is_string
        } kind;
        union
        {
          int ival;
          char *sval;
        } u;
      };
    }
    %define api.value.type {struct my_value}
    %token <u.ival> INT "integer"
    %token <u.sval> STRING "string"
    %printer { fprintf (yyo, "%d", $$); } <u.ival>
    %destructor { free ($$); } <u.sval>

    /* In yylex().  */
    yylval.u.ival = 42; return INT;
    yylval.u.sval = "42"; return STRING;

** Variable parse.error

  This variable controls the verbosity of error messages.  The use of the
  %error-verbose directive is deprecated in favor of "%define parse.error
  verbose".

** Renamed %define variables

  The following variables have been renamed for consistency.  Backward
  compatibility is ensured, but upgrading is recommended.

    lr.default-reductions      -> lr.default-reduction
    lr.keep-unreachable-states -> lr.keep-unreachable-state
    namespace                  -> api.namespace
    stype                      -> api.value.type

** Semantic predicates

  Contributed by Paul Hilfinger.

  The new, experimental, semantic-predicate feature allows actions of the
  form "%?{ BOOLEAN-EXPRESSION }", which cause syntax errors (as for
  YYERROR) if the expression evaluates to 0, and are evaluated immediately
  in GLR parsers, rather than being deferred.  The result is that they allow
  the programmer to prune possible parses based on the values of run-time
  expressions.

** The directive %expect-rr is now an error in non GLR mode

  It used to be an error only if used in non GLR mode, _and_ if there are
  reduce/reduce conflicts.

** Tokens are numbered in their order of appearance

  Contributed by Valentin Tolmer.

  With '%token A B', A had a number less than the one of B.  However,
  precedence declarations used to generate a reversed order.  This is now
  fixed, and introducing tokens with any of %token, %left, %right,
  %precedence, or %nonassoc yields the same result.

  When mixing declarations of tokens with a litteral character (e.g., 'a')
  or with an identifier (e.g., B) in a precedence declaration, Bison
  numbered the litteral characters first.  For example

    %right A B 'c' 'd'

  would lead to the tokens declared in this order: 'c' 'd' A B.  Again, the
  input order is now preserved.

  These changes were made so that one can remove useless precedence and
  associativity declarations (i.e., map %nonassoc, %left or %right to
  %precedence, or to %token) and get exactly the same output.

** Useless precedence and associativity

  Contributed by Valentin Tolmer.

  When developing and maintaining a grammar, useless associativity and
  precedence directives are common.  They can be a nuisance: new ambiguities
  arising are sometimes masked because their conflicts are resolved due to
  the extra precedence or associativity information.  Furthermore, it can
  hinder the comprehension of a new grammar: one will wonder about the role
  of a precedence, where in fact it is useless.  The following changes aim
  at detecting and reporting these extra directives.

*** Precedence warning category

  A new category of warning, -Wprecedence, was introduced. It flags the
  useless precedence and associativity directives.

*** Useless associativity

  Bison now warns about symbols with a declared associativity that is never
  used to resolve conflicts.  In that case, using %precedence is sufficient;
  the parsing tables will remain unchanged.  Solving these warnings may raise
  useless precedence warnings, as the symbols no longer have associativity.
  For example:

    %left '+'
    %left '*'
    %%
    exp:
      "number"
    | exp '+' "number"
    | exp '*' exp
    ;

  will produce a

    warning: useless associativity for '+', use %precedence [-Wprecedence]
     %left '+'
           ^^^

*** Useless precedence

  Bison now warns about symbols with a declared precedence and no declared
  associativity (i.e., declared with %precedence), and whose precedence is
  never used.  In that case, the symbol can be safely declared with %token
  instead, without modifying the parsing tables.  For example:

    %precedence '='
    %%
    exp: "var" '=' "number";

  will produce a

    warning: useless precedence for '=' [-Wprecedence]
     %precedence '='
                 ^^^

*** Useless precedence and associativity

  In case of both useless precedence and associativity, the issue is flagged
  as follows:

    %nonassoc '='
    %%
    exp: "var" '=' "number";

  The warning is:

    warning: useless precedence and associativity for '=' [-Wprecedence]
     %nonassoc '='
               ^^^

** Empty rules

  With help from Joel E. Denny and Gabriel Rassoul.

  Empty rules (i.e., with an empty right-hand side) can now be explicitly
  marked by the new %empty directive.  Using %empty on a non-empty rule is
  an error.  The new -Wempty-rule warning reports empty rules without
  %empty.  On the following grammar:

    %%
    s: a b c;
    a: ;
    b: %empty;
    c: 'a' %empty;

  bison reports:

    3.4-5: warning: empty rule without %empty [-Wempty-rule]
     a: {}
        ^^
    5.8-13: error: %empty on non-empty rule
     c: 'a' %empty {};
            ^^^^^^

** Java skeleton improvements

  The constants for token names were moved to the Lexer interface.  Also, it
  is possible to add code to the parser's constructors using "%code init"
  and "%define init_throws".
  Contributed by Paolo Bonzini.

  The Java skeleton now supports push parsing.
  Contributed by Dennis Heimbigner.

** C++ skeletons improvements

*** The parser header is no longer mandatory (lalr1.cc, glr.cc)

  Using %defines is now optional.  Without it, the needed support classes
  are defined in the generated parser, instead of additional files (such as
  location.hh, position.hh and stack.hh).

*** Locations are no longer mandatory (lalr1.cc, glr.cc)

  Both lalr1.cc and glr.cc no longer require %location.

*** syntax_error exception (lalr1.cc)

  The C++ parser features a syntax_error exception, which can be
  thrown from the scanner or from user rules to raise syntax errors.
  This facilitates reporting errors caught in sub-functions (e.g.,
  rejecting too large integral literals from a conversion function
  used by the scanner, or rejecting invalid combinations from a
  factory invoked by the user actions).

*** %define api.value.type variant

  This is based on a submission from Michiel De Wilde.  With help
  from Théophile Ranquet.

  In this mode, complex C++ objects can be used as semantic values.  For
  instance:

    %token <::std::string> TEXT;
    %token <int> NUMBER;
    %token SEMICOLON ";"
    %type <::std::string> item;
    %type <::std::list<std::string>> list;
    %%
    result:
      list  { std::cout << $1 << std::endl; }
    ;

    list:
      %empty        { /* Generates an empty string list. */ }
    | list item ";" { std::swap ($$, $1); $$.push_back ($2); }
    ;

    item:
      TEXT    { std::swap ($$, $1); }
    | NUMBER  { $$ = string_cast ($1); }
    ;

*** %define api.token.constructor

  When variants are enabled, Bison can generate functions to build the
  tokens.  This guarantees that the token type (e.g., NUMBER) is consistent
  with the semantic value (e.g., int):

    parser::symbol_type yylex ()
    {
      parser::location_type loc = ...;
      ...
      return parser::make_TEXT ("Hello, world!", loc);
      ...
      return parser::make_NUMBER (42, loc);
      ...
      return parser::make_SEMICOLON (loc);
      ...
    }

*** C++ locations

  There are operator- and operator-= for 'location'.  Negative line/column
  increments can no longer underflow the resulting value.