Location revisited, column tracking...

[Suodenjoki Michael Jari]

Hi,

 

I'm one of those developers that too would like Flex to handle locations (line/column tracking), so that e.g. error messages can be more user friendly.

 

As of today Flex can use the %option yylineno option to tell that the scanner should keep track of the line number (basically counting newlines). 

 

I've seen that this particular topic has been discussed in several threads in the last couple of months. Particular the January 2002 archive contains a huge thread on the subject.

 

I would personally find it useful to have an extra option e.g. named %option yylocation that tells the scanner to keep track of the location. If something like Akim's proposal in message http://mail.gnu.org/pipermail/help-flex/2002-January/000365.html could be implemented directly into Flex it would be nice.

 

I'm aware that it may be difficult to implement - but I dont think it's impossible. I'm also aware that even the todays implementation of %option yylineno may be considered faulty - depending on how you think it should work. The current documentation states nothing about that it will contain the line number of where the returned token was found!. The problem occurs e.g. in cases where the look ahead is forwarding over a new line character. But yet again this could be corrected at the same time (for the location at least).

 

So when a token is returned we (I) need some way of getting the token's location within the input source (start line, start column, end line and end column - should we also include start/end file/stream?). That's what I suggest that %option yylocation should be used for.

 

Regards,

/Michael

 

PS. By the way I would be more than happy to see or hear about other ways of implementing column tracking (or full location tracking).

Michael Suodenjoki
Software Engineer, Compliance Solutions Competence Center
WM-data
Lautrupvang 10
DK-2750 Ballerup
Denmark
Ph.: +45 4483 8436
Fax: +45 4483 8485
Email: address@hidden
Web: www.wmdata.com