Re: [Qemu-devel] [PATCH v11 22/28] qapi: Simplify visiting of alternate types

[Markus Armbruster]

Eric Blake <address@hidden> writes:

> Previously, working with alternates required two lookup arrays
> and some indirection: for type Foo, we created Foo_qtypes[]
> which maps each qtype to a value of the generated FooKind enum,
> then look up that value in FooKind_lookup[] like we do for other
> union types.
>
> This has a couple of subtle bugs.  First, the generator was
> creating a call with a parameter '(int *) &(*obj)->type' where
> type is an enum type; this is unsafe if the compiler chooses
> to store the enum type in a different size than int, where
> assigning through the wrong size pointer can corrupt data or
> cause a SIGBUS.
>
> Second, since the values of the FooKind enum start at zero, all
> entries of the Foo_qtypes[] array that were not explicitly
> initialized will map to the same branch of the union as the
> first member of the alternate, rather than triggering a desired
> failure in visit_get_next_type().  Fortunately, the bug seldom
> bites; the very next thing the input visitor does is try to
> parse the incoming JSON with the wrong parser, which normally
> fails; the output visitor is not used with a C struct in that
> state, and the dealloc visitor has nothing to clean up (so
> there is no leak).
>
> However, the second bug IS observable in one case: parsing an
> integer causes unusual behavior in an alternate that contains
> at least a 'number' member but no 'int' member, because the
> 'number' parser accepts QTYPE_QINT in addition to the expected
> QTYPE_QFLOAT (that is, since 'int' is not a member, the type
> QTYPE_QINT accidentally maps to FooKind 0; if this enum value
> is the 'number' branch the integer parses successfully, but if
> the 'number' branch is not first, some other branch tries to
> parse the integer and rejects it).  A later patch will worry
> about fixing alternates to always parse all inputs that a
> non-alternate 'number' would accept, for now this is still
> marked FIXME in the updated test-qmp-input-visitor.c, to
> merely point out that new undesired behavior of 'ans' matches
> the existing undesired behavior of 'asn'.
>
> This patch fixes the default-initialization bug by deleting the
> indirection, and modifying get_next_type() to directly assign a
> QTypeCode parameter.  This in turn fixes the type-casting bug,
> as we are no longer casting a pointer to enum to a questionable
> size. There is no longer a need to generate an implicit FooKind
> enum associated with the alternate type (since the QMP wire
> format never uses the stringized counterparts of the C union
> member names); that also means we no longer have a collision
> with an alternate branch named 'max'.  Since the updated
> visit_get_next_type() does not know which qtypes are expected,
> the generated visitor is modified to generate an error statement
> if an unexpected type is encountered.
>
> Callers now have to know the QTYPE_* mapping when looking at the
> discriminator; but so far, only the testsuite was even using the
> C struct of an alternate types.  I considered the possibility of
> keeping the internal enum FooKind, but initialized differently
> than most generated arrays, as in:
>   typedef enum FooKind {
>       FOO_KIND_A = QTYPE_QDICT,
>       FOO_KIND_B = QTYPE_QINT,
>   } FooKind;
> to create nicer aliases for knowing when to use foo->a or foo->b
> when inspecting foo->type; but it turned out to add too much
> complexity, especially without a client.
>
> There is a user-visible side effect to this change, but I
> consider it to be an improvement. Previously,
> the invalid QMP command:
>   {"execute":"blockdev-add", "arguments":{"options":
>     {"driver":"raw", "id":"a", "file":true}}}
> failed with:
>   {"error": {"class": "GenericError",
>     "desc": "Invalid parameter type for 'file', expected: QDict"}}
> (visit_get_next_type() succeeded, and the error comes from the
> visit_type_BlockdevOptions() expecting {}; there is no mention of
> the fact that a string would also work).  Now it fails with:
>   {"error": {"class": "GenericError",
>     "desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
> (the error when the next type doesn't match any expected types for
> the overall alternate).
>
> Signed-off-by: Eric Blake <address@hidden>
[...]
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index d4ef08e..1790e8f 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -627,15 +627,15 @@ def check_union(expr, expr_info):
>  def check_alternate(expr, expr_info):
>      name = expr['alternate']
>      members = expr['data']
> -    values = {'MAX': '(automatic)'}
> +    values = {}
>      types_seen = {}
>
>      # Check every branch
>      for (key, value) in members.items():
>          check_name(expr_info, "Member of alternate '%s'" % name, key)
>
> -        # Check for conflicts in the generated enum
> -        c_key = camel_to_upper(key)
> +        # Check for conflicts in the branch names
> +        c_key = c_name(key)
>          if c_key in values:
>              raise QAPIExprError(expr_info,
>                                  "Alternate '%s' member '%s' clashes with 
> '%s'"
> @@ -1090,8 +1090,11 @@ class QAPISchemaObjectTypeVariants(object):
>          assert isinstance(self.tag_member.type, QAPISchemaEnumType)
>          for v in self.variants:
>              v.check(schema)
> -            assert v.name in self.tag_member.type.values
> -            if isinstance(v.type, QAPISchemaObjectType):
> +            # Union names must match enum values; alternate names are
> +            # checked separately. Use 'seen' to tell the two apart.
> +            if seen:
> +                assert v.name in self.tag_member.type.values
> +                assert isinstance(v.type, QAPISchemaObjectType)
>                  v.type.check(schema)

Not exactly elegant, but it'll do.  Perhaps we can clean it up later.

>
>      def check_clash(self, schema, info, seen):
> @@ -1133,6 +1136,11 @@ class QAPISchemaAlternateType(QAPISchemaType):
>          # Not calling self.variants.check_clash(), because there's nothing
>          # to clash with
>          self.variants.check(schema, {})
> +        # Alternate branch names have no relation to the tag enum values;
> +        # so we have to check for potential name collisions ourselves.
> +        cases = {}
> +        for var in self.variants.variants:
> +            var.check_clash(self.info, cases)

If we call it cases here, perhaps we should call it members rather than
seen elsewhere.  I guess I'd simply stick to seen, though.

>
>      def json_type(self):
>          return 'value'
> @@ -1340,7 +1348,7 @@ class QAPISchema(object):
>          data = expr['data']
>          variants = [self._make_variant(key, value)
>                      for (key, value) in data.iteritems()]
> -        tag_member = self._make_implicit_tag(name, info, variants)
> +        tag_member = QAPISchemaObjectTypeMember('type', 'QTypeCode', False)

This drops generation of the implicit enum.

Only one user of _make_implicit_tag() left: _def_union_type() for simple
unions.  Possible followup: inline it there.

>          self._def_entity(
>              QAPISchemaAlternateType(name, info,
>                                      QAPISchemaObjectTypeVariants(None,
[...]