Re: Location revisited, column tracking...

[Hans Aberg]

At 17:26 +0200 2002/07/25, Akim Demaille wrote:
>In particular, I was pointing out that Flex does not know what the
>user calls `end of line', it can be \n, or \r\n, or \0, or several of
>them.

Or \r. Usually though, one opens a text stream, in which case it is always
\n. But then the character count goes out of synch, if it is \r\n. So one
way around that is to not program on platforms where it might be \r\n :-).

>  I was also pointing out that, in these Unicode days, Flex does
>not know what the user wants to track: characters or bytes.

And I was pointing out that the safest way around this is to assume single
width characters -- UTF-n, where n >= 24, letting the fellow that provides
the stream hook a code converter onto it (like C++ std::codecvt).

If somebody wants to make use of a variable width encoding, and get file
positions in bytes, that is going to be very complicated under any
circumstances, unless the OS provides some hooks.

Any way, I think that the problem of various Unicode encodings should be
handled by the stream writing fellows and not by Flex, because the issue is
much too complex.

  Hans Aberg