Re: scanner_cursor and yylloc

[Akim Demaille]

| Akim Demaille <address@hidden> writes:
| > It is not clear to me that we need both scanner_cursor and yylloc.
| 
| Surely we must have yylloc.  But if we get rid of scanner_cursor,
| won't yylex need to use yylloc as a read-write parameter?  In other
| words, won't yylex (&yylval, &yylloc) need to use yylloc.end as the
| initial value of scanner_cursor?

Yes, definitely.

| This raises a more general question.  I had thought that the usual
| tradition is that (yylex (&yylval, &yylloc)) writes to yylloc, but
| doesn't read from it, and similarly for yylval: i.e., the arguments to
| yylex are pointers to write-only storage.  However, I see that the
| Bison manual suggests using yylloc in the manner that you seem to be
| suggesting: i.e., that you can assume that a parser always invokes a
| lexer with the same yylloc every time.

Yes, that is true.

| Is this a wise assumption in general?  For example, doesn't the GLR
| parser use multiple yylloc values in some cases?  If so, we shouldn't
| rely on the parser passing the address of the same yylloc variable to
| the lexer every time, we should redo the examples in the Bison manual
| that assume otherwise, and we should continue to separate
| scanner_cursor from yylloc.
| 
| A less important reason to keep the two separate is that
| unexpected_end_of_file inserts the expected token (to avoid cascading
| messages), and to do this it needs to subtract from scanner_cursor
| without subtracting from yylloc.

I see.  Actually, I never quite understood why you add the missing
terminator, I usually do this:

<STATE_STRING>{ /* Handling of the strings.  Initial " is eaten. */
     \" {
       BEGIN INITIAL;
       return STRING;
     }

// [...]
     <<EOF>> {
       std::cerr
         << *yylloc << ": unexpected end of file in a string" << std::endl;
       exit_set (exit_scan);

       BEGIN INITIAL;
       return STRING;
     }
}

I don't see what rescanning brings.




Anyway, you do make a point.  My problem is bound to the fact that at
least the initial yylloc must come from the user, and given that it is
yyparse that the user calls, it seems logical to me that yylloc can be
read from the scanner.  And that an initial location must be
provided.