Re: PMT thoughts

[Josh Morman]

John,

Thank you for sharing the issues you have come across and recommendation for a cleaned up API.  As Johannes mentioned, the GREP highlights our current thinking on PMTs also by leveraging a flatbuffers backend for standardized serialization.  There is currently a proof of concept implementation of this in the "newsched" codebase which is our sandbox for developing the GNU Radio 4.0 runtime:  https://github.com/gnuradio/newsched/tree/master/pmt and we would love to have any feedback on the GREP and help pushing the concept further.

Thanks,

Josh

On Fri, Mar 26, 2021 at 5:44 AM Johannes Demel <demel@ant.uni-bremen.de> wrote:

Hi John,

you're not alone with your concerns. There's a GREP (GNU Radio
Enhancement Proposal) [0] where issues with PMTs are discussed. This
would be an excellent place to discuss more details and contribute.

Cheers
Johannes

[0] https://github.com/gnuradio/greps/pull/31

On 25.03.21 23:43, John Sallay wrote:
> I appreciate the effort that you guys have put into cleaning up gnuradio
> and making it easier to work with over the past few years (python2 -
> python3, swig to pybind11).  I see that you are going to improve the
> logging as well which is great.
> I've recently been doing some message based processing, and been using
> PMTs.  I think that PMTs could use an overhaul too.
> Here are some of my concerns:
> The interface is inconsistent.  Here are a few examples:
> is_integer, but from/to_long
> is_dict, is_vector, not no is_list
> make_dict, make_vector, but list1, list2, etc
>
> Type safety doesn't really work in several cases:
> # apt install of gr 3.9 on ubuntu 20.04
> a = pmt.make_dict()
> a = pmt.dict_add(a, pmt.intern("a"), pmt.intern("a"))
> pmt.is_dict(a) # True
> a = pmt.list_add(a, pmt.intern("b"))   # I believe that this changes the
> dictionary to a list
> pmt.is_dict(b) # False
> I could come up with more examples, but they illustrate roughly the same
> point.  I should note that something changed in GR3.9.  It was even
> worse in 3.8.  For example, on a list, pmt.is_dict() returns True.  I
> can call some of the dict functions which return unexpected results,
> while others raise an error.
>
> The interface doesn't feel like modern C++.  The pmt object doesn't
> really have any useful functions associated with it.  You have to use
> the functions in pmt namespace to operate on any pmt.  I believe that
> PMTs are using boost::any under the hood.  I can get why some of the
> choices were made, but I think that there may be some alternatives
> available that could lead to an easier interface.
>
> I've been playing around with std::variant (in c++17, boost::variant
> prior to that).  I have some ideas for how we could use it to wrap the
> scalar types as well as the vector and dynamic types.  I would be happy
> to explore the idea if there is interest.  We could also use variadic
> templates to avoid things like list1, ..., list6.
>
> Using std::variant, I would envision an interface like:
> pmt my_dict();
> my_dict["abc"] = 4.0 // Automatically convert the string and the number
> to the proper type.
> std::get<float>(my_dict["abc"]))
>
> pmt_list my_list();
> my_list.append(pmt(my_dict));
> for (auto& element : my_list) {
>     // Do something
> }
> std::cout << my_list.type() << " " << my_list << std::endl;
>
> We should be able to deprecate, but maintain the current functions for
> dealing with PMTs.  I believe that these changes could be made without
> breaking existing code.
>
> To be clear, I haven't implemented any of this.  My goal right now is
> just to see if there is interest in doing something like this.  If there
> is, I am happy to look into it and try to create a proof of concept. 
> I'm also open to suggestions and feedback.
>
> Thanks,
>
> John
>