Re: [Qemu-devel] [RFC] QCFG: a new mechanism to replace QemuOpts and option handling

[Anthony Liguori]

On 03/22/2011 08:01 AM, Markus Armbruster wrote:
> Type checking macros are feasible (see [*] for an existence proof), but
> things do get hairy, and the resulting error messages can be less than
> clear at times.

That just gives you a warning.  You can do much better things with 
__builtin_types_compatible_p.

But neither really solves the problem I'm talking about.  I can go into 
it in depth but hopefully we both can agree that trying to build 
introspection macros is pretty difficult if not impossible :-)

> >                                                              However,
> > this makes it very difficult to support things like lists of lists or
> > anything else that would basically require a non-concrete type.
> Sounds like you want a more expressive type system than C's, and propose
> to get it by building your own DSL.  I'm not sure that's wise.

It's an IDL.  IDL based RPCs are pretty common with C.  The IDL is 
purely declarative.

> > If you plan to expose these types in a library, you need to either
> > explicitly pad each structure and make sure that the padding is
> > updated correctly each time a new member is added.
> As long as the data description is data, writing a program to check that
> a new version is compatible with the old one shouldn't be hard.

If you define the structs on your own, you need to either have a data 
description of the padding or be very careful doing it yourself.

> >                                                      Alternatively, you
> > can add an allocation function that automatically pads each structure
> > transparently.
> Weaker than a comprehensive check, but could be good enough.
>
> > qmp-gen.py creates qmp-types.[ch] to do exactly the above and also
> > generates the type declaration so that you don't have to duplicate the
> > type marshalling code and the type declaration.  Today, this is close
> > to 2k LOCs so it's actually a significant amount of code code.
> >
> > There is also the code that takes the input (via QCFG or QMP) and
> > calls an appropriate C function with a strongly typed argument.  I've
> Not sure I got you here.  Perhaps an example could enlighten me :)

void qapi_free_vnc_info(VncInfo *obj)
{
    if (!obj) {
        return;
    }
    if (obj->has_host) {
        qemu_free(obj->host);
    }
    if (obj->has_family) {
        qemu_free(obj->family);
    }
    if (obj->has_service) {
        qemu_free(obj->service);
    }
    if (obj->has_auth) {
        qemu_free(obj->auth);
    }
    if (obj->has_clients) {
        qapi_free_vnc_client_info(obj->clients);
    }


    qapi_free_vnc_info(obj->next);
    qemu_free(obj);
}

It's pretty basic boiler plate code that could be written by hand, but 
why not generate it.  It actually all adds up pretty quickly in terms of 
SLOCs.

> > The mechanism I described using the visitor pattern is really the
> > right solution for vmstate.  The hard problems to solve for vmstate
> > are:
> >
> > 1) How to we support old versions in a robust way.  There are fancy
> > things we could do once we have a proper visitor mechanism.  We could
> > have special marshallers for old versions, we could treat the output
> > of the visitor as an in memory tree and do XSLT style translations,
> > etc.
> >
> > 2) How do we generate the visitor for each device.  I don't think it's
> > practical to describe devices in JSON.  It certainly simplifies the
> > problem but it seems ugly to me.  I think we realistically need a C
> > style IDL and adopt a style of treating it as a header.
> Now I'm confused.  Do you mean your JSON-based DSL won't cut it for
> vmstate?
>
> If yes, why is it wise to go down that route now?

There are a few paths we could go.  We can describe devices in JSON.  
This makes VMState introspectable with all of the nice properties of 
everything else.  But the question is, do we describe the full device 
state and then use a separate mechanism to cull out the bits that can be 
recomputed.

To we only describe the guest visible state and treat that as a separate 
structure?  Is that embedded in the main state object or do we 
explicitly translate the main state object to this new type?

We can pretty easily have a C-like IDL so I'm not terribly concerned 
about describing devices in JSON.  It's about finding the right way to 
structure device marshalling in the long term.

So yes, I think JSON is the right approach, but that doesn't mean I've 
figured out how to do VMState.

> > Yeah, this is one of the big challenges with vmstate.  There needs to
> > be a clearer line between internal types and object models vs. the
> > externally visible interfaces.
> Not only with vmstate.  If we couple QMP closely to internal interfaces,
> I'm afraid we'll end up in the same unhappy place we're now with
> vmstate.

Yeah, it's a tough balance to strike.  If you expose too much detail 
about internals, it's very difficult to maintain compatibility in the 
long term.  If you have a pure external interface, it's difficult to 
make sure that the external interfaces are actually useful because 
there's no immediate feedback mechanism.

Regards,

Anthony Liguori

> > Regards,
> >
> > Anthony Liguori
> >
> > > [...]
> [*] http://ccan.ozlabs.org/info/check_type.html