Re: why are locations dictated by bison?

[Hans Aberg]

At 14:28 -0800 2002/01/11, John W. Millaway wrote:
>As maintainer of Bison, you have strongly opposed implementing location
>tracking in flex, and you believe this should be left to the user to define.
>What business, then, does Bison have dictating the format of the location
>tracking? Why is location tracking a parser issue?

I think that Akim is thinking of something more complicated that what I
(and we?) suggested. I of coruse let Akim spcify that part.

>The result of Bison's YYLTYPE mechansim is that the scanner needs to know
>about
>parser details. This is arguably bad design -- The scanner should be
>completely
>separate from the parser -- according to some. With that in mind, bison should
>simply leave out location tracking altogether, shouldn't it?

The picture that I have in my mind is that the lexer can first specify
positions. A position might consist of one or more of the following data:
- Line number, optionally with column number and additionally pointer to
the line beginning.
- Stream position.
- Stream descriptor/structure/name.
Plus perhaps some other such data.

A location is then a [begin, end) position pair. The Flex lexer provides a
means to find the location data of the yytext token, and that is it.

The second question is how to get that stuff onto the parser. The variation
that generates the least lexer overhead is to put into that info raw into
the parser. For specialized error displays, the variation that causes the
least overhead on the lexing is to process this data already available at
error time.

If Flex is tuned together with Bison's default YYLTYPE structure, then
first I think one should change the Bison default YYLTYPE so it can match
what Flex can produce: I suspect that Bisons YYLTYPE is an old one, rarely
used.

But even with such a redesigned YYLTYPE in place, the data actually put
into yyltype may differ for special tokens. So Flex should have some way to
change that in the actions of the token rules it concerns.

Then, on the same time, some may want to design completely different
location tracking schemes. Then one should be able to do that, merely using
the location data of yytext that Flex provides if useful, and one may then
want to design a customized YYLTYPE as well.

  Hans Aberg