Re: [PATCH 3/5] c++: improvements on symbol kinds

[Hans Åberg]

> On 15 Apr 2020, at 18:25, Akim Demaille <address@hidden> wrote:
> 
>> Le 15 avr. 2020 à 18:15, Hans Åberg <address@hidden> a écrit :
>>>>> In retrospect, it would have been better to have it named "token_type", 
>>>>> and "stack_symbol_type" would have been "symbol_type".
>>>> 
>>>> Maybe so. I only need the symbol value for the return to the parser from 
>>>> the lexer, and mostly, they are symbol constants (tokens), but I am not 
>>>> sure they always have to be.
>>> 
>>> So that's the kind.
>> 
>> I think it is unkind! :-) The word “kind” does not seem to be in line with 
>> any C++ lingo at least.
> 
> The parser is fed with tokens by yylex.  A lexeme such as "123" in an 
> arithmetical expression, has:
> 
> - a kind, NUMBER.  That coincides with what people refer to as a "token" when 
> they talk about the grammar.
> 
> - a value, the int 100 + 20 + 3.
> 
> - a location, from Line 1 Col 1 to Line 1 Col 4 (excluded)
> 
> The type of the kind is token_kind_type, the type of the value is value_type, 
> and the type of the location is location_type.  The type of the kind is not 
> kind_type, because we actually deal with two kinds, so we must be clearer.  
> But there's only one concept of value_type and one of location_type.
> 
> And token_type denotes the type which is a triple: token_kind_type, 
> value_type, location_type.
> 
> yylex returns the kind, and takes pointers to yylval (value) and yylloc 
> (location).

Currently, I return the token value, and the semantic and location values are 
passed through the argument. It is more logical to have them in the return, but 
bundling them together do not make them tokens, they are parser tokens bundled 
together with other data.

Perhaps it might be the called lexeme value.