Re: [Qemu-devel] [PATCH v10 10/13] qapi: Don't box struct branch of alternate

[Markus Armbruster]

Eric Blake <address@hidden> writes:

> On 02/16/2016 12:07 PM, Markus Armbruster wrote:
>> Eric Blake <address@hidden> writes:
>> 
>>> There's no reason to do two malloc's for an alternate type visiting
>>> a QAPI struct; let's just inline the struct directly as the C union
>>> branch of the struct.
>>>
>>> Surprisingly, no clients were actually using the struct member prior
>>> to this patch; some testsuite coverage is added to avoid future
>>> regressions.
>>>
>>> Ultimately, we want to do the same treatment for QAPI unions, but
>>> as that touches a lot more client code, it is better as a separate
>>> patch.  So in the meantime, I had to hack in a way to test if we
>>> are visiting an alternate type, within qapi-types:gen_variants();
>>> the hack is possible because an earlier patch guaranteed that all
>>> alternates have at least two branches, with at most one object
>>> branch; the hack will go away in a later patch.
>> 
>> Suggest:
>> 
>>   Ultimately, we want to do the same treatment for QAPI unions, but as
>>   that touches a lot more client code, it is better as a separate patch.
>>   The two share gen_variants(), and I had to hack in a way to test if we
>>   are visiting an alternate type: look for a non-object branch.  This
>>   works because alternates have at least two branches, with at most one
>>   object branch, and unions have only object branches.  The hack will go
>>   away in a later patch.
>
> Nicer.
>
>> 
>>> The generated code difference to qapi-types.h is relatively small,
>>> made possible by a new c_type(is_member) parameter in qapi.py:
>> 
>> Let's drop the "made possible" clause here.
>
> I was trying to document the new is_member parameter somewhere in the
> commit message, but I can agree that it could be its own paragraph
> rather than a clause here.

That could work.

>> This is in addition to visit_type_BlockdevOptions(), so we need another
>> name.
>> 
>> I can't quite see how the function is tied to alternates, though.
>> 
>
> I'm open to naming suggestions.  Also, I noticed that we have
> 'visit_type_FOO_fields' and 'visit_type_implicit_FOO'; so I didn't know
> whether to name it 'visit_type_alternate_FOO' or
> 'visit_type_FOO_alternate'; it gets more interesting later in the series
> when I completely delete 'visit_type_implicit_FOO'.
>
>>>
>>> and use it like this:
>>>
>>> |     switch ((*obj)->type) {
>>> |     case QTYPE_QDICT:
>>> |-        visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
>>> |+        visit_type_alternate_BlockdevOptions(v, name, 
>>> &(*obj)->u.definition, &err);
>> 
>> Let's compare the two functions.  First visit_type_BlockdevOptions():
>> 
>
>> 
>> Now visit_type_alternate_BlockdevOptions(), with differences annotated
>> with //:
>> 
>>     static void visit_type_alternate_BlockdevOptions(Visitor *v,
>>                                     const char *name,
>>                                     BlockdevOptions *obj, // one * less
>
> Yep, because we no longer need to malloc a second object, so we no
> longer need to propagate a change to obj back to the caller.
>
>>                                     Error **errp)
>>     {
>>         Error *err = NULL;
>> 
>>         visit_start_struct(v, name, NULL, 0, &err); // NULL instead of &obj
>>                                                     // suppresses malloc
>>         if (err) {
>>             goto out;
>>         }
>>         // null check dropped (obj can't be null)
>>         visit_type_BlockdevOptions_fields(v, obj, &err);
>
> Also, here we pass 'obj'; visit_type_FOO() had to pass '*obj' (again,
> because we have one less level of indirection, and 7/13 reduced the
> indirection required in visit_type_FOO_fields()).
>
>>         // visit_start_union() + switch dropped
>>         error_propagate(errp, err);
>>         err = NULL;
>>         visit_end_struct(v, &err);
>>     out:
>>         error_propagate(errp, err);
>>     }
>> 
>> Why can we drop visit_start_union() + switch?
>
> visit_start_union() is dropped because its only purpose was to determine
> if the dealloc visitor needs to visit the default branch. When we had a
> separate allocation, we did not want to visit the branch if the
> discriminator was not successfully parsed, because otherwise we would
> dereference NULL.  But now that we don't have a second pointer
> allocation, we no longer have anything to dealloc, and we can no longer
> dereference NULL. Explained better in 12/13, where I delete
> visit_start_union() altogether.  But maybe I could keep it in this patch
> in the meantime, to minimize confusion.

Perhaps squashing the two patches together could be less confusing.

> Dropped switch, on the other hand, looks to be a genuine bug.  Eeek.
> That should absolutely be present, and it proves that our testsuite is
> not strong enough for not catching me on it.
>
> And now that you've made me think about it, maybe I have yet another
> idea.  Right now, we've split the visit of local members into
> visit_type_FOO_fields(), while leaving the variant members to be visited
> in visit_type_FOO()

Yes.  I guess that's to support visiting "implicit" structs.

> visit_type_FOO_fields() is static, so we can change it without impacting
> the entire tree; I could add a bool parameter to that function, and write:
>
> visit_type_FOO() {
>   visit_start_struct(obj)
>   visit_type_FOO_fields(, false)
>   visit_end_struct()
> }
>
> visit_type_FOO_fields(, wrap) {
>   if (wrap) {
>     visit_start_struct(NULL)
>   }
>   visit local fields
>   visit variant fields
>   if (wrap) {
>     visit_end_struct()
>   }
> }
>
> and for the variant type that includes FOO as one of its options,
>
> visit_type_VARIANT() {
>   allocate (currently spelled visit_start_implicit_struct, but see 13/13)
>   switch (type) {
>   case QTYPE_QINT:
>     visit_type_int(); break
>   case QTYPE_QDICT:
>     visit_type_FOO_fields(, true); break
>   }
>   visit_end
> }
>
> or not even create a new function, and just have:
>
>
> visit_type_FOO() {
>   visit_start_struct(obj)
>   visit_type_FOO_fields()
>   visit_end_struct()
> }
>
> visit_type_FOO_fields() {
>   visit local fields
>   visit variant fields
> }
>
> and for the variant type,
>
> visit_type_VARIANT() {
>   allocate (currently spelled visit_start_implicit_struct, but see 13/13)
>   switch (type) {
>   case QTYPE_QINT:
>     visit_type_int(); break
>   case QTYPE_QDICT:
>     visit_start_struct(NULL)
>     visit_type_FOO_fields()
>     visit_end_struct()
>     break
>   }
>   visit_end_implicit
> }
>
> where the logic gets a bit more complicated to do correct error handling.

Let me get to this result from a different angle.  A "boxed" visit is

    allocate hook
    visit the members (common and variant)
    cleanup hook

An "unboxed" visit is basically the same without the two hooks.

"Implicit" is like unboxed, except the members are inlined rather than
wrapped in a JSON object.

So the common code to factor out is "visit the members".

>>> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
>>> index 2f23432..aba2847 100644
>>> --- a/scripts/qapi-types.py
>>> +++ b/scripts/qapi-types.py
>>> @@ -116,6 +116,14 @@ static inline %(base)s *qapi_%(c_name)s_base(const 
>>> %(c_name)s *obj)
>>> 
>>> 
>>>  def gen_variants(variants):
>>> +    # HACK: Determine if this is an alternate (at least one variant
>>> +    # is not an object); unions have all branches as objects.
>>> +    inline = False
>>> +    for v in variants.variants:
>>> +        if not isinstance(v.type, QAPISchemaObjectType):
>>> +            inline = True
>>> +            break
>>> +
>>>      # FIXME: What purpose does data serve, besides preventing a union that
>>>      # has a branch named 'data'? We use it in qapi-visit.py to decide
>>>      # whether to bypass the switch statement if visiting the discriminator
>>> @@ -136,7 +144,7 @@ def gen_variants(variants):
>>>          ret += mcgen('''
>>>          %(c_type)s %(c_name)s;
>>>  ''',
>>> -                     c_type=typ.c_type(),
>>> +                     c_type=typ.c_type(is_member=inline),

I don't like the name is_member.  The thing we're dealing with here is a
member, regardless of the value of inline.  I think it's boxed
vs. unboxed.

>>>                       c_name=c_name(var.name))
>>> 
>>>      ret += mcgen('''
[...]
>>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>>> index 4d75d75..dbb58da 100644
>>> --- a/scripts/qapi.py
>>> +++ b/scripts/qapi.py
[...]
>>> @@ -984,8 +984,10 @@ class QAPISchemaObjectType(QAPISchemaType):
>>>          assert not self.is_implicit()
>>>          return QAPISchemaType.c_name(self)
>>>
>>> -    def c_type(self, is_param=False):
>>> +    def c_type(self, is_param=False, is_member=False):
>>>          assert not self.is_implicit()
>>> +        if is_member:
>>> +            return c_name(self.name)
>>>          return QAPISchemaType.c_type(self)
>> 
>> To understand this, you have to peel off OO abstraction:
>> QAPISchemaType.c_type(self) returns c_name(self.name) + pointer_suffix.
>> 
>> I think we should keep it peeled off in the source code.
>
> By this, you mean I should do:
>
> def c_type(self, is_param=False, is_member=False):
>     assert not self.is_implicit()
>     if is_member:
>         return c_name(self.name)
>     return c_name(self.name) + pointer_suffix
>
> ?

Yes, that's what I had in mind.

> or that I should hoist the suppression of pointer_suffix into the parent
> class?

c_type()'s is_param is ugly, and is_member adds to the ugliness.  If you
can come up with a clean way to avoid one or both, go for it.

>>>      def json_type(self):
>> [tests/ diff looks good]
>
> good for what they caught, but they didn't catch enough :)