Re: ending ; (Was: POSIX_ME_HARDER)

[Paul Eggert]

Akim Demaille <address@hidden> writes:

> Look at the code!  Gosh, it hurts!

Most of the patch in
<http://mail.gnu.org/archive/html/bison-patches/2002-12/msg00003.html>
has to do with improving the quality of locations in Bison
diagnostics.  (The previous code didn't accurately identify the starts
of tokens and comments.)  These changes do make the code more
complicated, but I didn't see a way to simplify it while still having
accurate line and character numbers in diagnostics.  No doubt I should
have separated this into a different patch, but I was rushed and the
two issues are intermixed I'm afraid.

I think the part of the code that you're concerned about is the
<SC_AFTER_IDENTIFIER> part in scan-gram.l.  But this is only 20 or 30
lines.  I don't think that such a small amount of code is much of a
cost to support the POSIX requirement.


> I don't even understand why we did not move to the GLR parser, which
> is precisely _the_ tool for these kind of non LR(1)-ness.

If the GLR parser solves the problem in a better way, then that would
be great.  However, my impression was that the GLR parser is still a
bit experimental, particularly when it comes to handling syntax errors
and the like.  I don't know of any production code that uses it yet.
So I didn't want to rely on it in a conservative patch to a stable
branch.  But if the GLR parser works well it'd be nice to use it in
the next major release (after 2.0, that is).


> you even wanted to support the full feature including the most
> unlikely complications such as
> 
>       foo /* A first comment \
>       on\
>        several lines*/ // and even some C++ comment!
>       :

The support for those comments between "foo" and ":" is exactly the
same Lex code as the support for those comments between any other two
grammar symbols.  There is no code duplication, so I don't see why
this would be considered an extra complexity.

Anyway, POSIX requires support for C-style comments in that position,
and such comments occur in real grammars, e.g. the Gawk grammar.  So
that patch had to continue to support them.


> Now, as far as extensions are concerned, the lack of this ;,
> supporting that it is missing, will cause us a lot of trouble.

I don't see why it will cause us much trouble.  The only potential
extension that you've mentioned so far is the ability to have rules
like "a b: c d;".  I don't recall what you wanted that syntax for, but
surely it's easy to support that extended syntax for grammars that put
a ";" after every rule, while at the same time supporting the
POSIX-required syntax for grammars that omit ";" after rules.  And
even if I'm wrong and it's not easy, it will be easy enough to put a
keyword in front of the extended rules; that will remove any
ambiguity.  We could use "%rule a b: c d;", say.