Re: Bison 3.2.90 released [beta]

[Derek Clegg]

> On Jan 14, 2019, at 11:56 PM, Akim Demaille <address@hidden> wrote:
> 
> Hi Derek,
> 
>> Le 13 janv. 2019 à 23:40, Derek Clegg <address@hidden> a écrit :
>> 
>> It appears that “api.prefix” changes the definition of YYSTYPE:
>> 
>> %define api.prefix {cmap_yy}
>> …
>> union CMAP_YYSTYPE
>> …
>> 
>> I don’t know whether this is a new change or not. I switched to “api.prefix” 
>> due to this error:
>> 
>> cmap-parser.y:71.1-12: error: deprecated directive, use ‘%define api.prefix’ 
>> [-Werror=deprecated]
>> %name-prefix "cmap_yy"
>> ^^^^^^^^^^^^
>> 
>> Unfortunately, the corresponding option in flex does not have the same 
>> behavior: flex-generated files, even with the corresponding flex option set
>> 
>> %option prefix="cmap_yy"
>> 
>> refer to YYSTYPE, not CMAP_YYSTYPE, at least as of flex-2.5.35.
> 
> I don't think flex's scanner depend on YYSTYPE.

It does if you specify %option bison-bridge in your flex file. I think this is 
an unfortunate case where flex is relying on long-standing bison practice.

> Flex does not know anything about yylval and yylloc.  In fact you have to 
> define YY_DECL to tell flex what the signature of ylex is, and therefore the 
> names and types of the arguments for the value and the location.
> 
> So I venture that somewhere you have something like
> 
> #define YY_DECL int yylex (YYSTYPE* yylval, YYLTYPE* yylloc)
> 
> which should become
> 
> #define YY_DECL int cmap_yylex (CMAP_YYSTYPE* yylval, CMAP_YYLTYPE* yylloc)

While this is correct, it doesn’t fix the problem when %option bison-bridge is 
specified.

> 
>> This makes it not non-trivial to set an API prefix for flex-generated and 
>> bison-generated files.
> 
> I agree the change is more demanding than just updating the grammar file, you 
> also need to adjust YY_DECL.  I personally put YY_DECL in the %code provides 
> section, so in the end I still only need to update the grammar file, but your 
> situation might be different.
> 
> I am unhappy that this change requires more changes, however, I really 
> believe that the previous situation was wrong: if you have several parsers, 
> %name-prefix allows symbols in the objects to not collide, yet you cannot 
> include their headers in a single compilation unit, as the types would 
> collide.  IMHO, %name-prefix was buggy.

I agree with this point. Unfortunately, the feature as it stands doesn’t work 
without changes to my flex sources, which seems undesirable.

> 
> I should have provided some doc for this.  What do you think about the 
> following patch?
> 
> diff --git a/doc/bison.texi b/doc/bison.texi
> index c66a9746..5a724a98 100644
> --- a/doc/bison.texi
> +++ b/doc/bison.texi
> @@ -6650,6 +6650,28 @@ For example, if you use @samp{%define api.prefix 
> @address@hidden, the names become
> @code{cparse}, @code{clex}, @dots{}, @code{CSTYPE}, @code{CLTYPE}, and so
> on.
> 
> +Users of Flex must update the signature of the generated @code{yylex}
> +function.  Since the Flex scanner usually includes the generated header of
> +the parser (to get the definitions of the tokens, etc.), the most convenient
> +way is to insert the declaration of @code{yylex} in the @code{provides}
> +section:
> +
> address@hidden
> +%define api.prefix @address@hidden
> +// Emitted in the header file, after the definition of YYSTYPE.
> +%code provides
> address@hidden
> +  // Tell Flex the expected prototype of yylex.
> +  #define YY_DECL                             \
> +    int clex (CSTYPE*y ylval, CLTYPE *yylloc)
> +
> +  // Declare the scanner.
> +  YY_DECL;
> address@hidden
> address@hidden example
> +
> address@hidden 1
> +
> The @code{%define} variable @code{api.prefix} works in two different ways.
> In the implementation file, it works by adding macro definitions to the
> beginning of the parser implementation file, defining @code{yyparse} as
> 

I think this is good, but there still needs a way to deal with %option 
bison-bridge in my opinion.

Derek