Re: Accepting GCJ's grammar

[Akim Demaille]

| > From: Akim Demaille <address@hidden>
| > Date: 10 Jun 2002 18:10:42 +0200
| > 
| > GCJ does a weird thing:
| > 
| > | %{
| > | #include "lex.c"
| > | %}
| > | ...
| > | %token   PLUS_TK         MINUS_TK        MULT_TK         DIV_TK    REM_TK
| > | ...
| > 
| > With the funny thing that lex.c uses PLUS_TK etc.
| 
| This won't work with Yacc, which outputs a #define for a token as it
| sees the %token directive.  So, GCJ apparently assumes Bison and not
| Yacc.

Yes, it does.

| It is fairly easy to fix GCJ here, is it not?  I.e. I assume that this
| is merely a backward compatibility issue, not an issue of an important
| Bison feature.

the tokens very early, before any user prologue, CVS Bison output the
tokens after the two prologues.  They relied on 1.35's behavior, but
as is, they can't have it with CVS Bison.

The easy solution is to separate the scanner for the compiler's
compilation unit.  Or, to have the scanner be #included, but to
include the parser's header right before.

| POSIX doesn't seem to require any particular order for the C output.
| The only thing I could find on the topic is that it is explicitly
| unspecified whether the programs section (the stuff after the 2nd %%)
| is output after or before the code corresponding to the grammar-rule
| actions.  Clearly there are some informal constraints here; the most
| natural assumption, I would think, is that code should be output in
| the same order that it was input.

Right.  But it doesn't fit very well the M4 model here.

| > I.e., we do try to provide the user with a means to disable
| > YYTOKENTYPE via the #ifndef (well, that's my reading, but maybe that's
| > not what you meant Paul.  Maybe it's merely to cope with include
| > "parser.h" in parser.c).
| 
| I meant the former, but I guess the latter is important too.

It is.  And it's not completely trivial to have the parser being able
to include its header.  WIth 1.28 it's impossible, and in most
application I checked, it results in redefinitions.  Plus the order of
defaulting (#ifndef YYDEBUG then #define YYDEBUG) in the header
collides with that in the parser.

In other words, I see no point in making this impossible, but it's a
dangerous game.


| > Which means we cannot output the tokens in the pre-prologue.  So, to
| > have GCJ compile properly, I propose the following patch, which output
| > the token defs in the post-prologue.
| 
| I guess that's OK, though it seems to me that the real fault is with
| GCJ here.

I tend to agree.

| > I don't apply it, because as is, it does not allow one to write:
| > 
| >        %{
| >        void yyerror (const char *s);
| >        int yylex (void);
| >        #ifndef MY_TOKEN
| >        # error "MY_TOKEN not defined."
| >        #endif
| >        %}
| >        %token MY_TOKEN
| >        %%
| >        exp: MY_TOKEN;
| >        %%
| 
| I don't think we need to support that, any more than we need to
| support the GCJ style anyway.  Both styles are wrong: a .y file
| shouldn't refer to a token name before the %token directive.

Well, I insist, as it is, neither work.  I share your idea of `natural
order', but it's unrealistic.  For instance, should

%token foo
%{
  using foo
%}
%token bar

result in YYTOKENTYPE's enum be defined in several pieces?  I don't
know how to do that.  I don't know if all the languages we will aim at
support this.

So really, I do not want to have to handle such things.


| > So, to summarize: my current idea is what expresses this patch, which
| > invalidates the snippet right above.  If your plan does not provide
| > the user with the right to touch YYTOKENTYPE, then we can simply
| > output token defs very early and ignore all this issue.
| 
| I thought it might be useful for the user to define YYTOKENTYPE.  I
| don't see why that isn't still possible, so long as it's done before
| the %union.

In CVS Bison it is possible, and in my proposal it still is before the
%union.  There seems to be some misunderstanding here.


| However, this is becoming quite hacky.  Ideally we would modify Bison
| so that the output is in the same order as the input.  This would
| require some changes to the templates (i.e., get rid of
| b4_token_defines, and have a different scheme for outputting each
| token definition when it's read).  But it'd be much easier to document
| and to explain.

Yes.  Plus, I don't think this idea is well defined.

I'm applying the patch.