Re: Locations suggest

[Akim Demaille]

>>>>> "Hans" == Hans Aberg <address@hidden> writes:

>> Some people will tell you they need (c0, l0) + wdith, not (c0, l0)
>> + (c1, l1).  And you cannot convert from one to the other without
>> having the text handy.

Hans> I am not sure what you mean here: The width is just the
Hans> difference between the position numbers; see the thread
Hans> "Flex/Bison standardized diagnostics".

Wrong, you're thinking only of the case where everything is on a
single line.


Hans> I think that it would be interesting with an option that allows
Hans> any of \n, \r, \r\n to be read as a newline, so that streams can
Hans> be read as binary.

Hans> Unicode has a special line separator symbol, so then tha issue
Hans> may go away.

No, it's not another issue: it's the same.  This idea, IMO, is fine,
but cannot face the difference between uses.

>> - dependent upon the lexical structure of the input (tell me how
>> you expect to handle tabs: single char?  Walks by 4 spaces? by 8?)

Hans> Tabs is a non-issue in the proposed set-up, that is, programmers
Hans> will have to implement it on their own.

Of course it is!!!

Hans> But one might give the implementation of a tab = whitespace
Hans> equivalence a thought, as that is used in some programs (for
Hans> example, in Haskell). Then one must have a variable telling how
Hans> many whites there is on a tab. One extra problem is that C/C++
Hans> whitespace [[:space:]]+ has several different tabs in
Hans> it. Unicode, I think, has several fixed width tabs, so the issue
Hans> may pop up again in that context.

Thanks for reexplaining what I explained.

Hans> But my own experience is that one should avoid giving such an
Hans> equivalence in parsers, as the number may vary in usage.

Good, you seem to agree on my point.


>> - dependent upon the taste of the user.

Hans> There are no such dependencies: The proposal provides the
Hans> information needed for any programmer to customize.

Sorry, I meant the Flex user, who is indeed the programmer.


>> All we need a means to implement what we need. We already have it.

Hans> It does not seem so, not even for you: You posted an example
Hans> with extremely complicated code for something which should be
Hans> fairly transparent to the programmer.

It's not complicated at all.  If you understand Flex, you understand
the code.  I managed to do what I needed to, that's the conclusion.
This is Flex, not ToyFlex, it aims at programmers.

Now, better sugar for implementing these bits would be fine, but then,
you departure from POSIX.  Still, I'm OK with some sugar, but
definitely having Flex imagining it can perform the job for me is
wrong.

>> All the rest is pushing an additional burden onto you, for
>> extremely limited added value.

Hans> Isn't that for the Flex developers remaining to see? If one is
Hans> curious about a problem, perhaps giving it an attempt, then that
Hans> will tell exactly what is required to implement it.

Have _you_ tried?  I did.  I implemented many scanners with location
tracking, and they all have different needs, especially because they
have different ``blanks''.


>> In addition, it is more expensive to have Flex do it for us: it
>> duplicates the walks through yytext.

Hans> What do you have in your mind here? No such backtracking is
Hans> needed in any of the discussion that we have had here.

You wrote backtracking, I did not.

Just read the code, you'll see what I mean.  If you never read the
code, then there is no point in debating.

Hans> Clearly, if Flex implements it properly, that will be the most
Hans> efficient variation, as Flex has access to things that the user
Hans> does not.

No, the user knows better than Flex the lexical structure of the file
it parses.

Hans> The problem appears to be that Flex has some quick fixes (as the
Hans> REJECT code implementation) which slows it down.

That's another issue, buggy IMO.  I'm talking about the implementation
of option yylineno itself.