Re: [Qemu-devel] Re: [RFC][PATCH v1 05/12] qapi: fix handling for null-return async callbacks

[Anthony Liguori]

On 03/28/2011 12:59 PM, Michael Roth wrote:
> > > For a command like this, I can't imagine ever wanting to extend the
> > > return value...
>
>
> I think this is another topic, but also one we should hash out a bit 
> better.
>
> Currently the plan is that the C API not expose asynchronicity, 
> underneath the covers the library will issue the request, then do a 
> blocking read for the response. So the API call will block till 
> completion, and no other command's will be serviced through the same 
> underlying session until it is completed or cancelled.

No, that's just the patches as they stand today.

The medium term goal is to have twin APIs--one API that is synchronous 
and another that is asynchronous.  The asynchronous version will mirror 
how asynchronous commands are dispatched within QEMU.  That is, for 
query-block, you'll have:

typedef void (*QueryBlockFunc)(void *opaque, BlockInfo *retval, Error *err);

void libqmp_async_query_block(QmpSession *sess, Error **errp, 
QueryBlockFunc *cb, void *opaque);

The challenge with async library commands is that you need to think 
carefully about how you interact with the socket.  You can make a glib 
mainloop be a prerequisite, or you can have a set of function pointers 
that let you implement your own main loop.

But since there isn't a pressing need for this in the short term, it's 
easier to just delay this.

> For the JSON-based clients, the behavior is different. You have an 
> optional tag you can pass in for an async command, and after issuing 
> one, you can immediately begin issuing other async or non-async 
> commands. As a result, the responses you receive will not necessarily 
> be in FIFO order.

There is no such thing as a "JSON-based client".  QMP is not self 
describing enough to implement this with a pure transport client.  
Another language implementation (which I think you mean) would still 
need to solve the same problem as libqmp.

> The upside to this is you can implement async commands on the client 
> side without using separate threads, and can exploit some level of 
> concurrency being able to do work within a session while a slower 
> host->guest command completes. The downsides are that:
>
> 1) There is some inconsistency between this and the C API semantics.

You still need to pass a continuation to implement an event-based 
interface regardless of the language you're using.  The function 
pointer/opaque parameter is a continuation in C.

> 2) The "optional" tags are basically required tags, at least for async 
> commands, unless the client side does something to force synchronicity.
>
> One option would be to disable the QMP session's read handler once a 
> JSON object is received, and not re-enable it until we send the 
> response. This would enforce FIFO-ordering. It might also add reduce 
> the potential for a client being able to blow up our TX buffers by 
> issuing lots of requests and not handling the responses in a timely 
> enough manner (have seen this just from piping responses to stdout).

No, we're mixing up wire semantics with client/server semantics.

These are all completely different things.

The wire semantics are:

1) All commands are tagged.  Untagged commands have an implicit tag 
(let's refer to it as the psuedo-tag).

2) Until a command is completed, a tag cannot be reused.  This is also 
true for the psuedo-tag.

3) There is no guarantee about command completion order.

4) If a client happens to use the same tag for all commands, the client 
ends up enforcing a completion order because the server is only ever 
processing one command at a time.

The server semantics are:

1) All tags are preserved including the psuedo-tag.  This is required by 
the protocol.

2) Most commands are implemented by immediately dispatching a function 
and then computing the return value and immediately putting on the 
socket buffer.

3) Some commands are implemented by delaying the computation of the 
return value.  When this happens (which is an indeterminate amount of 
time later), the data will be put on the socket buffer.

4) Which commands are handled by (2) vs. (3) are transparent to the client.

The (current) client semantics are:

1) All commands are tagged with the psuedo-tag which enforces that only 
one command can be in flight at a time.  In the future, this interface 
will support threading and use different tags such that two threads can 
be used to send simultaneous commands.

2) A second interface will be implemented that provides an event-based 
interface whereas each command is passed a continuation.  Commands will 
use different tags to support this interface.

3) The reason to have both interfaces is to support the two most common 
models of concurrency, event-based concurrency and thread based concurrency.

Notice that I said nothing about 'C' in the above.  It's equally true to 
a C or Python client.

Regards,

Anthony Liguori

> Thoughts?