Re: RFC: enum instead of #define for tokens

[Akim Demaille]

>>>>> "Hans" == Hans Aberg <address@hidden> writes:

Hans> At 12:19 +0200 2002/04/04, Akim Demaille wrote: You are right,
Hans> this is not very nice :-):
>> IMHO, my position is not nice wrt people who are abusing the
>> system.

Hans> It is not abuse; it is standard Yacc. You removed yourself the
Hans> original --raw feature that skipped the characters.

I would appreciate if you avoided to propagate false statement.  I did
remove --raw, which impact was to numbering from 3 instead of 257.


Hans> Right know, I do not know how scanner errors are handed over to
Hans> the parser -- perhaps there should be a macro for that in the
Hans> .tab.h header.

Everybody out of range is mapped to $undefined.



>> All this discussion is anyway not taking into account the impact
>> that Unicode can have on the size of the Bison tables.  From the
>> theory point of view, I'm very much against Unicodization, from the
>> practical point of view, I'm not even sure it is doable.  And most
>> importantly, I'm sure that if it's done in the scanner, these
>> problems vanish.

Hans> I am not sure what you are speaking about here: The regular
Hans> token numbers are translated, so they will not have any impact
Hans> by Unicode at all.

Where do you put them?  What's the impact on the size of the tables?
Plus all the needed stuff to deal with the different encodings of
Unicode.  Much worse than what Paul highlighted wrt say EBCDIC.


>> Yacc and Lex, as Parsers and Scanners, struggled from a clean
>> separation of the two different tasks.  We should keep these task
>> separate.

Hans> This is a purist point of view: Yacc and Lex are not theoretical
Hans> tools, but something that should be convenient to the
Hans> user. Besides, if you merge the scanner/parser language
Hans> hierarchy, from the theoretical point of view, you get a single
Hans> language.

Which is what I had detailed in the following paragraph, so that we
don't have to debate about this point.


>> Note that this is very different from referring the input
>> formalism.  We can very well imagine a FlexBison that input a
>> single file for both the scanner and the parser.  But still there
>> would be a parser and a scanner in the output.  So called
>> scannerless parsers do have a separation somewhere between lexical
>> and syntactic.

Hans> I am not speaking about a Flex/Bison merge, here. 

Me neither.