Re: Using bison without Flex

[John Levine]

In article <address@hidden> you write:
>
>> On 23 Apr 2015, at 16:34, brahim sahbi <address@hidden> wrote:
>
>> can we use bison without flex by describing grammatical
>> rules for tokens(like number <- list_of_digits).
>
>A lexer like generated by Flex scans forward to find the longest match which 
>may require significant lookahead. A Bison
>generated parser just looks ahead at most one token.
>
>So in theory yes, but in practise not really.

You could have a stub yylex that returned individual characters and
assemble them using bison rules.  In languages with reasonable
tokenization rules (not Fortran) that can work.

>> Can it recognize a language's keywords and prohibit an identifier to be a
>> keyword?

You might have to resolve that using GLR, or the hack that a
reduce/reduce conflict is resolved in favor of the earlier rule in the
bison script:

whilekwd: 'w' 'h' 'i' 'l' 'e' ;
ifkwd: 'i' 'f' ;
thenkwd: 't' 'h' 'e' 'n' ;
elsekwd: 'e' 'l' 's' 'e' ;

identifier: letter | identifier letter ;
letter: 'a' | 'b' | 'c' ... 'z' ;

The real reason you don't want to do this is that practical lexers
invariably have extra rules invisible to the parser to throw away
white space and comments.  Putting them into the parser is extremely
unpleasant because you end up having to put an
optional_comment_or_white_space after every token in your rules.

R's,
John