[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Dynamic token kinds

From: Frank Heckenbach
Subject: Re: Dynamic token kinds
Date: Mon, 17 Dec 2018 18:37:56 +0100

Hans Åberg wrote:

> > What I suggest to add (without changing the above), is:
> > 
> >  symbol_type make_symbol (token_type type);
> >  // checks at runtime that type is V_FOO or V_BAR
> > 
> >  symbol_type make_symbol (token_type type, int &&);
> >  // checks at runtime that type is I_BAZ or I_QUX
> > 
> >  symbol_type make_symbol (token_type type, string &&);
> >  // checks at runtime that type is S_BLA
> > 
> > These runtime checks might be implemented via a switch if that's
> > easier to auto-generate (it might be in fact) or with a simple
> > "if (... || ...)" statement, that's an implementation detail.
> Actually, I pass the semantic value through a class object to
> which the lexer function belongs, so the extra arguments are not
> needed. So I must think more about this.

Do you actually use Bison's variant? Otherwise, what you do is
irrelevant here (sorry), as this is a proposal specifically about
Bison's variant.

> >>> It's not that bad actually. Again, my lexers work fine as is.
> >>> I just brought this up because Akim proposed to call the function
> >>> "unsafe_..." which I thought was too harsh and proposed
> >>> "unchecked_..." -- but by adding the checks, it would be neither
> >>> unsafe nor unchecked. :)
> >> 
> >> This worries me.
> > 
> > That's why I suggest to add the check. :)
> There must be some guard against programming errors, I think.

That's why I suggest to add the check. :)

> >> But also having having to use something more complex to be returned by the 
> >> lexer than a value on the lookup table .
> > 
> > The lexer returns a token which contains the token kind (an enum)
> > and the semantic value (a union value). As mismatch is bad. The
> > make_FOO functions avoid a mismatch and are suitable for statically
> > known token kinds. The direct constructor call can be used for
> > dynamic token kinds, but allows a mismatch. The functions I propose
> > to generate instead could be used for dynamic token kinds and avoid
> > a mismatch.
> > 
> > Everything clear now?
> Yes, it is the requirement of returning the semantic value that
> causes the issue. Then this requirement is perhaps born out the
> condition of not storing the type in the Bison variant.

Building rather than returning. And sure, it's because Bison's
variant doesn't store the type, we know this.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]