Re: Porting to typed C++ parser (was: Dynamic token kinds)

[Hans Åberg]

> On 18 Dec 2018, at 13:20, Frank Heckenbach <address@hidden> wrote:
> 
> Hans Åberg wrote:
> 
>>> On 17 Dec 2018, at 19:09, Frank Heckenbach <address@hidden> wrote:
>> 
>> [Note: you don't cc me, only others, for some reason.]
> 
> By default, I keep recipients (list) and CCs (Akim) as is.

With a reply to all, it would come out right.

> If you
> want a private copy, set a Reply-To.

Those are generally bad. I might not notice your posts, so do as you wish.

>>>> Actually, I pass the semantic value through a class object to
>>>> which the lexer function belongs, so the extra arguments are not
>>>> needed. So I must think more about this.
>>> 
>>> FWIW, my lexer function is also part of a "driver" object, but I
>>> don't think that's relevant here. Maybe you think of the "%param"
>>> arguments, but it's not about those.
>> 
>> No, I use the Flex C++ parser right now, which for some reason
>> requires an empty argument, so this works without patching the
>> header, that is, until they decided to bomb it in 2.6 and later.
> 
> I don't use flex anymore, and never used flex C++, so I don't know
> about that. But again, that's unrelated to the current proposal.

It was in response to your comment.

>>> In my proposal:
>>> 
>>> symbol_type make_symbol (token_type type, int &&);
>>> 
>>> the second parameter is actually the semantic value (and the first
>>> one the token kind, of course), so there are no extra arguments, no
>>> driver, no lexer, not even the parser (no hidden "this" parameter,
>>> since these are static functions), so I think there's nothing to
>>> worry about. It's just about building a token from these two
>>> parameters, as you'd expect to, by basically calling its
>>> constructor, and (as per the proposal) adding checks to help avoid
>>> mismatches between the two of them.
>>> 
>>> But if you tell us more about how you (plan to) build your tokens,
>>> we could say if there are any potential problems.
>> 
>> I use a polymorphic (virtual) pointer class, a string, a token
>> number, or a combination thereof depending on context. For the
>> first, it might be useful with typecasts that simplify the code.
> 
> A variant is (in this regard) somewhat similar to a polymorphic
> pointer (except that the pointer needs memory management, as you
> know), so it seems you should be able to use a variant instead.
> 
> With variants, you usually don't use type-casts, but accessors, such
> as get<> with std::variant or as<> with Bison's variant, though in
> Bison parsers, that usually happens automatically with $n.
> 
> So I see no reason why you shouldn't be able to use variants, but of
> course, without seeing any details, I can just talk generally.

There might be a problem with the Bison variant if one in the lexer returns an 
object of another type that the token type, and expect a conversion.