Re: rename lex_symbols (was: Rename variant and lex_symbols options)

[Akim Demaille]

Hi Joel, Hi all,

This is a very old thread; I really want to close it in the very near
future.  Basically, finding proper names for %define variant and
%define lex-symbols is what prevents the release of the master branch.
The issue is to find appropriate names, and be sure that we don't lock
ourselves in some badly chosen user interface. Currently they are only
implemented for C++, but one could imagine other skeletons to offer
similar features.

Previous discussions with Joel have established that they are not
related, so let's treat them separately.  Simplest first: %define
lex-symbols.

This variable allows to generate functions that construct "symbols":
objets that binds together the kind, the semantic value, and possibly
the location.  That's only visible from the scanner, so instead of
writing

      yylval->ival = value;
      *yylloc = location_type ();
      return yy::parser::token::NUMBER;

one writes

      return yy::parser::make_NUMBER (value, location_type ());

Note that this way the scanner is type-checked (make_NUMBER requires
that "value" is an int, whereas in the first case, since yylval is
just a union, we can do nonsensical assignments), and initialization
is checked for every token (you _must_ provide a value for NUMBER,
whereas in the first case, if you don't assign *yylval at all, nothing
will be reported (see the thread with Paul about GCC warnings for
yylval being not initialized).  Note also that while this is
implemented for C++ only (both lalr1.cc and glr.cc), there is nothing
that prevents its use in other languages.

Since this has a direct influence on the API, it clearly belongs to
the api.* pseudo-namespace of %define variables.  Currently, they are:

  %define api.location.type
  %define api.namespace
  %define api.prefix
  %define api.pure
  %define api.push-pull
  %define api.token.prefix

The "token" subcategory seems to fit nicely.  Something like

  %define api.token.constructor

would make sense.  Yet Joel reminded us that we would like to follow
two rules of thumb: avoid Boolean variables, and avoid plurals (so
it's api.token, not api.tokens).  But I don't see well what names
would allow use to avoid a mere Boolean value.

If someone has a better idea, I'd be happy to read it!