Re: problem linking program containing bison-generated parser

[Laurence Finston]

On Fri, 11 Mar 2005, Frank Heckenbach wrote:

> Laurence Finston wrote:
>
> > I've found that I prefer using Flex and Bison separately rather than
> > together.  If you're just starting out learning Bison, I think you
> > might find it easier to write your own `yylex()' function rather than
> > using Flex.
>
> Actually why? For lexing identifiers, numbers, the usual stuff, it's
> one line in flex, and loops, sometimes switches and manual
> loop-ahead in C.
> And for similar-beginning tokens (e.g. `&', `&&',
> `&=' in C), IMHO it gets more readable in flex (one entry per item,
> can be group topically -- arithmetics, comparisons, ...) than in
> manual code (where they have to be handled together after reading
> the first `&').

> So I'm interested why you find the manual way easier.

I had problems generating a reentrant scanner using Flex,
and I wasn't able to find out what was going wrong.  There
was some discussion about this on this list at the time,
about a year ago.

In my scanner, I use what I call the "category code" approach
rather than regular expressions.  It's the way Knuth used
for TeX and Metafont.  I'm pretty sure that
any scanner written in one of these ways could be reformulated
using the other one.  However, I find that the category code
approach allows for finer control.  At least one other person has
had problems with regular expressions and posted a question
to this list since I've been a subscriber.

It might not be possible to implement a couple of things that
my scanner does in a straightforward way with Flex.
Therefore, I think that using Flex might artificially
limit the kinds of things people do with Bison.
I don't recall the details, since it's been awhile since
I've worked on this part of my program.

Nor do I think that the power of Flex is needed for the kind of
scanning Bison needs.  I like to use Flex when I can implement
functionality in its actions, rather than just passing tokens
to Bison.

If someone wants to learn Bison, I think it might simplify the task
if he or she could just concentrate on Bison, rather than having to
learn Flex at the same time.  I also think that implementing `yylex()'
oneself might give one a better insight into what it does and
what Bison's requirements are.  This might help one to understand how
to use Flex well in combination with Bison, if one later decides to
use it.

I try to avoid using `switch' (I just don't like it), and I don't have
to examine the lookahead token too often.  Otherwise, I like
doing the sort of somewhat low-level programming you describe.

Clearly, this is a matter of taste, and I certainly don't think
there's anything wrong with using Bison in combination with Flex,
if someone prefers to do this.  I might even do it myself, if
I ever really needed regular expressions when scanning.  However,
I thought it was worthwhile to point out that it isn't necessarily
the best way under all circumstances.

Laurence