scanner_cursor and yylloc

[Paul Eggert]

Akim Demaille <address@hidden> writes:

> It is not clear to me that we need both scanner_cursor and yylloc.

Surely we must have yylloc.  But if we get rid of scanner_cursor,
won't yylex need to use yylloc as a read-write parameter?  In other
words, won't yylex (&yylval, &yylloc) need to use yylloc.end as the
initial value of scanner_cursor?

This raises a more general question.  I had thought that the usual
tradition is that (yylex (&yylval, &yylloc)) writes to yylloc, but
doesn't read from it, and similarly for yylval: i.e., the arguments to
yylex are pointers to write-only storage.  However, I see that the
Bison manual suggests using yylloc in the manner that you seem to be
suggesting: i.e., that you can assume that a parser always invokes a
lexer with the same yylloc every time.

Is this a wise assumption in general?  For example, doesn't the GLR
parser use multiple yylloc values in some cases?  If so, we shouldn't
rely on the parser passing the address of the same yylloc variable to
the lexer every time, we should redo the examples in the Bison manual
that assume otherwise, and we should continue to separate
scanner_cursor from yylloc.

A less important reason to keep the two separate is that
unexpected_end_of_file inserts the expected token (to avoid cascading
messages), and to do this it needs to subtract from scanner_cursor
without subtracting from yylloc.