Re: Suggestions for C++ parser generator

[Mark Boyall]

In addition: If you change the filename type to a wstring, then compilation
fails because the stream operators are hardcoded to output to a narrow
stream. Fix: template them on the stream type.

On 4 August 2011 17:40, Mark Boyall <address@hidden> wrote:

> I've been playing with the Variant you posted me, and it's a big
> improvement. I found it vastly easier to express complex logic in the
> parser. I wanted to point out a couple of issues that I had:
>
> Firstly, the Variant won't build on Visual Studio 2010, because the
> assignment operator doesn't return a value. I simply added return *this; to
> silence the compiler.
>
> Secondly, Bison states that the default action for a rule is { $$ = $1; }.
> However, this is not exactly true- when $1 is of a type that is implicitly
> convertible to $$, for example via inheritance, Bison will issue a redundant
> warning. Simply by adding { $$ = $1; } I invoked the implicit conversion and
> removed the warning.
>
> Thirdly, I attempted to update the Variant class to support move semantics
> and there is an issue you should be aware of if you ever intend to support
> them. The problem is that when using rvalue references, you have to pass the
> type explicitly if you don't want it to be perfectly forwarded- and since
> there are implicit conversions from a reference, this causes the type to be
> deduced as a reference for perfect forwarding which cannot be placed in the
> buffer. The solution here is to always use a factory function and always
> explicitly pass the intended target type. I presume that, since older
> compilers don't support rvalue references, if you were to choose to support
> them it would have to be conditional flagged support.
>
> For example,
>
>     template <typename T>
>     inline T&
>     build (const T& t)
>     {
>       return *new (buffer.raw) T(t);
>     }
>
> should become
>
>     template <typename T, typename Arg>
>     inline T&
>     build (Arg&& a)
>     {
>       return *new (buffer.raw) T(std::forward<Arg>(a));
>     }
>
> where T is always explicitly provided. Secondly, implicit constructions
> should be removed- only empty Variants should be allowed to be constructed
> and then build used on them.
>
> On the plus side, it was mostly hassle-free to use rvalue references- as a
> mutable reference is provided when using $1 for example then it was easy to
> use std::move as appropriate.
>
> On 28 July 2011 16:03, Mark Boyall <address@hidden> wrote:
>
>>  Actually, the parser was not really meant to be subclassed.  Maybe that
>>> was an error, agreed.
>>
>>
>> In addition, the yyerror() function could be pure virtual. Adding
>> functions into libraries like this is what virtual functions are for :) It
>> would also alleviate the need for yylex() to take additional arguments in
>> quite a few cases, because those arguments could be just data members of the
>> derived class.
>>
>> The latest versions of Visual Studio and GCC both support move semantics,
>> as does Clang and Comeau- basically all of the major compilers, as far as I
>> know, now support rvalue references in their latest versions and sometimes
>> going back a considerable way.
>>
>> Actually it is plain standard C++ (it's a deque), and it's useful.  What
>>> is painful though is that it is a separate file.  I'd like to merge it into
>>> the header file once for all (but backward compat will have to be maintained
>>> in some way).
>>
>>
>> Sure, but you'd still have to update it, and configure it, for if the
>> compiler supports rvalue references. If you just used a deque directly then
>> you would not have to worry about it.
>>
>>
>>> Yes, indeed, but that's a whole new project you asking for here.  Bison
>>> is a parser generator, not yet a parser-and-ast generator.
>>
>>
>> That's probably true. I don't actually know that much about the Bison
>> implementation- it's full of tables and stuff and I don't understand it
>> tremendously well. What I really should have said is that I found it a
>> slight pain when dealing with recursive rules to build data structures of
>> those rules, which maintain the order they should do- especially when
>> multiple recursive rules is invoked separately and repeatedly in the same
>> construct. The problem with the existing functional programming style is
>> that you can't pass anything *forwards*, only backwards- unless you're
>> feeling brave with the minus numbers. One of the things I could suggest is
>> instead of using goto to jump between states, you could use mutually
>> recursive functions. The advantage of this is that firstly, each function
>> can be strongly typed without the need for a hold-all union or variant, and
>> secondly, each function can be given user-defined additional arguments on a
>> per-function basis. Of course, I don't know how much effort it would be to
>> enable such a design.
>>
>> The variant design you post about would be a massive improvement and solve
>> a lot of my problems relating to using Bison. Right now I am using a
>> (relatively) complex arrangement in my user-defined argument to hold the
>> correct data structure for various recursive rules, and it's quite complex
>> and hard to reason about. I'm using Bison 2.5 and don't see anything about a
>> %variant option. I will look through the link you posted and see what I can
>> make happen with it.
>>
>> On 27 July 2011 17:42, Akim Demaille <address@hidden> wrote:
>>
>>>
>>> Le 26 juil. 2011 à 17:17, Mark Boyall a écrit :
>>>
>>> Hi!
>>>
>>> > I've got a couple of simple suggestions for the C++ parser generator.
>>> >
>>> > Firstly, yylex() should be declared as a (pure) virtual function in the
>>> > parser interface. This would make using it re-entrantly significantly
>>> easier
>>> > and would mean that the user does not have to declare it.
>>>
>>> Actually, the parser was not really meant to be subclassed.  Maybe that
>>> was an error, agreed.
>>>
>>> > Secondly, stack.hh should be replaced with the Standard stack class.
>>> This
>>> > will ensure that users who have move support can use move-only classes
>>> when
>>> > interacting with Bison.
>>>
>>> Actually it is plain standard C++ (it's a deque), and it's useful.  What
>>> is painful though is that it is a separate file.  I'd like to merge it into
>>> the header file once for all (but backward compat will have to be maintained
>>> in some way).
>>>
>>> Where do you use move semantics?  Are you using some C++-11 compiler?
>>>
>>> > Thirdly, I suggest that Bison should generate it's own AST classes,
>>> with
>>> > inheritance being used to link rules which have alternatives. This
>>> would
>>> > solve, mostly, the problem of the semantic types only being POD data
>>> types.
>>>
>>> Yes, indeed, but that's a whole new project you asking for here.  Bison
>>> is a parser generator, not yet a parser-and-ast generator.
>>>
>>> Have you experienced the %variant mode in Bison development version?  It
>>> works well, and avoids this nasty POD issue.  Have a look at
>>> http://www.lrde.epita.fr/~akim/download/bison-2.4.570-7a582.tar.bz2.
>>>  Look for variant in the documentation.  The following example shows what
>>> you can do.  I need to polish and publish this :(  Help would be most
>>> welcome.
>>>
>>> %type <::std::string> item;
>>> %type <::std::list<std::string>> list;
>>>
>>> %%
>>>
>>> result:
>>>  list  { std::cout << $1 << std::endl; }
>>> ;
>>>
>>> list:
>>>  /* nothing */ { /* Generates an empty string list */ }
>>> | list item     { std::swap ($$, $1); $$.push_back ($2); }
>>> ;
>>>
>>> item:
>>>  TEXT          { std::swap ($$, $1); }
>>> | NUMBER        { $$ = string_cast ($1); }
>>> ;
>>> %%
>>>
>>>
>>>
>>
>