Re: [PATCH v14 02/14] fixup! migration: New QAPI type 'MigrateAddress'

[Daniel P . Berrangé]

On Fri, Oct 20, 2023 at 02:37:16PM +0200, Markus Armbruster wrote:
> Fabiano Rosas <farosas@suse.de> writes:
> 
> > Markus Armbruster <armbru@redhat.com> writes:
> >
> >> Fabiano Rosas <farosas@suse.de> writes:
> >>
> >>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> >
> > Side question: are we using valid JSON at all? I threw this in a random
> > online linter and it complains about the single quotes. We could have a
> > proper tool doing the validation in CI.
> 
> You've come a sad, sad place.
> 
> docs/devel/qapi-code-gen.rst:
> 
>     Schema syntax
>     -------------
> 
>     Syntax is loosely based on `JSON <http://www.ietf.org/rfc/rfc8259.txt>`_.
>     Differences:
> 
>     * Comments: start with a hash character (``#``) that is not part of a
>       string, and extend to the end of the line.
> 
>     * Strings are enclosed in ``'single quotes'``, not ``"double quotes"``.
> 
>     * Strings are restricted to printable ASCII, and escape sequences to
>       just ``\\``.
> 
>     * Numbers and ``null`` are not supported.
> 
> If your reaction to item 2 is "this is stupid", you'd be exactly right.
> 
> Here's the conclusion of a discussion on possible improvements we had in
> 2020:
> https://lore.kernel.org/qemu-devel/877dt5ofoi.fsf@dusky.pond.sub.org/

Looking at those options again I so strongly want to be able to
say "none of the above".

We have a need to describe data structures, and generate code for
serializing/deserializing them on the wire. We created a language
for this and wrote our own C code generator, our own docs generator,
own our json parser & formatter, and now are also writing our own
Go code generator (Python, Rust too ?).

IMHO this is way too much NiH for my likely, and creates a maint
burden for ourselves that we could do without.

<open-can-of-worms>
At the point in time we invented QAPI this was perhaps justifiable,
though even back then I thought we should have done a binary format
and used XDR to describe it, as a standard pre-existing language and
toolset.

Today I wouldn't suggest XDR, but would be inclined to suggest
Protobuf. This already has code & docs generators for every
programming language we would ever need.

Protobuf has a defined serialization format too though which is
binary based IIUC, so distinct from our JSON wire format.

The interesting question though is whether it would be feasible to
transition to this today, despite our historical baggage ? With
recent interest in accessing QAPI for many more languages, it
is timely to consider if we can adopt a standardized toolset to
reduce this burden of talking to QEMU from other languages.

A transition would need several big ticket items

 * A QAPI visitor impl that can spit out the protobuf document.
   ie to translate all our existing QAPI formats into protobuf,
   as doing this manually would be madness. This is probably
   the easy bit.

 * A custom protobuf visitor impl that can spit out C code
   that has the same API as our QAPI C generator. ie so we can
   avoid a big bang to convert all of QEMU internals. I suspect
   this is quite alot of work.

 * A custom protobuf serializer compatible with our current
   JSON format. ie so that QEMU can continue to expose the
   current QMP protocol to apps for legacy compat for some
   period of time, while also exposing a new native binary
   protobuf protocol for future usage. Also probably quite
   alot of work.

That's all certainly quite alot of work and probably things I have
forgotten. Above all it does assume it is possible to define a
loss-less mapping from QAPI -> Protobuf language. I've not proved
this is possible, but my inclination is that it probably should be,
and if not, we could likely enable it by strategically deprecated
and then deleting troublesome bits prior to a conversion.
</open-can-of-worms>

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|