Re: [Qemu-devel] [PATCH v2] Show values and description when using "qom-list"

[Markus Armbruster]

"Dr. David Alan Gilbert" <address@hidden> writes:

> * Markus Armbruster (address@hidden) wrote:
>> "Dr. David Alan Gilbert" <address@hidden> writes:
>> 
>> > * Markus Armbruster (address@hidden) wrote:
>> >> Markus Armbruster <address@hidden> writes:
>> >> 
>> >> > Andreas Färber <address@hidden> writes:
>> >> >
>> >> >> Am 08.06.2018 um 11:41 schrieb Dr. David Alan Gilbert:
>> >> >>> * Andreas Färber (address@hidden) wrote:
>> >> >>>> Am 01.06.2018 um 17:39 schrieb Ricardo Perez Blanco:
>> >> >>>>> For debugging purposes it is very useful to:
>> >> >>>>>  - See the description of the field. This information is already 
>> >> >>>>> filled
>> >> >>>>>    in but not shown in "qom-list" command.
>> >> >>>>
>> >> >>>> No objection on this part.
>> >> >>>>
>> >> >>>>>  - Display value of the field.
>> >> >>>>
>> >> >>>> That is by definition the qom-get operation, not qom-list. Just like 
>> >> >>>> the
>> >> >>>> ls command does not show file contents, there's cat etc. for that. 
>> >> >>>> For
>> >> >>>> debugging purposes we had a qom-tree (?) command that would combine
>> >> >>>> both.
>> >> >>>
>> >> >>> I'm not too bothered about distinguishing between the two commands;
>> >> >>> but it would be nice
>> >> >
>> >> > When an HMP and QMP both have a command with the same name, they should
>> >> > do the same.
>> >> >
>> >> > HMP may add convenience features that aren't wanted in QMP, but I feel
>> >> > extending an operation to list objects to also show their contents goes
>> >> > beyond that.  If we want an HMP command that does both, it should be
>> >> > named differently.  Perhaps that might even be more appropriate for HMP
>> >> > than low-level commands qom-list and qom-get, but I leave that to the
>> >> > HMP maintainer to decide.
>> >> >
>> >> >>>                      - one reason I'm not too bothered is because 
>> >> >>> we've
>> >> >>> failed to get a qom-get in multiple years of trying.
>> >> >
>> >> > We clearly haven't tried hard enough.
>> >> >
>> >> > If we can figure out how to show values in qom-list, surely we can
>> >> > figure out how to show them in qom-get.
>> >> >
>> >> >>>>       There might be unmerged patches on qemu-devel related to 
>> >> >>>> display
>> >> >>>> of certain data types.
>> >> >>> 
>> >> >>> Which ones?
>> >> >>
>> >> >> My original qom-info series needed StringOutputVisitor changes for 
>> >> >> enums
>> >> >> (test case: rtc) that did not get accepted immediately and thus some
>> >> >> part of HMP qom-info/qom-get got stuck due to risking assertions for
>> >> >> qom-info / otherwise; QMP was not affected IIRC.
>> >> >
>> >> > Here's the last try I can find:
>> >> > [PATCH v2] qom: Implement qom-get HMP command
>> >> > Message-Id: <address@hidden>
>> >> > https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg01041.html
>> >> 
>> >> Stalled on output format and consistency with qom-set.  I wrote back
>> >> then "We can take qom-get as is and improve its output later."  I'd like
>> >> to encourage you to dust it off.  Perfect's the enemy of good.
>> >> 
>> >> Wanted improvements include:
>> >> 
>> >> * Prettier output format.  I'd suggest creating a keyval variant of the
>> >>   output visitor.
>> >
>> > I'm not too bothered about pretty at first, as long as we don't stop
>> > anyway of making it pretty later; especially if non-compound types work
>> > well.
>> 
>> We've waited so long for "someone" to post a solution that satisfies all
>> wants.  We should take a solution that is useful and can grow.
>> 
>> >> * Make qom-set input format consistent by switching to the matching
>> >>   input visitor.
>> >> 
>> >> > Its v1 tries a different approach:
>> >> > [PATCH 0/2] qom-get [for 2.8]
>> >> > Message-Id: <address@hidden>
>> >> > Unfortunately the mailing list archive doesn't show the full thread, so
>> >> > you get to follow three links:
>> >> > https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg03815.html
>> >> > https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04261.html
>> >> > https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04267.html
>> >> 
>> >> This one stalled on string visitor limitations.  You didn't feel like
>> >> addressing them just to get qom-get working.  Understandable.
>> >
>> > Have there been any changes in the last 2 years that have helped?
>> 
>> There's been progress, but there are gaps left.
>> 
>> Back then, we had the choice of string input/output visitors, and the
>> options visitor, all (severely) restricted to certain kinds of data.
>> 
>> Now we have the qobject keyval input visitor, which is almost general.
>> There is no matching output visitor, yet, simply because we haven't had
>> the need.
>
> That's potentially a big help; is:

Before I answer your questions, let me reiterate: perfect's the enemy of
good.  Perfect is what has denied us qom-get in HMP.

>    a) Does keyval output here make sense (It seems reasonable, perhaps
>       with some wrapping etc)

I guess it wouldn't be my first choice on a green field.  But this field
is anything but green.

The data structure we need to print is a recursive QObject.  QObject is
basically a JSON AST.  One obvious way to print it is JSON.
Comptetitors in that space include YAML and TOML.

Unlike these three, "keyval" isn't recursive.  Instead, it uses dotted
keys to express the nesting.  Moreover, mapping from JSON to "keyval"
can lose information (see "bridge the gap" below).

But then CLI uses "keyval" syntax extensively, and HMP is related to CLI
(e.g. object_add's argument is like -object's argument).  Use of the
same format for printing information and for reading configuration is
generally appreciated by users.

QMP does that by using JSON for both.

qom-get and qom-set could do that by using "keyval" for both.

>    b) Is this the code in util/keyval.c + qapi/qobject-input-visitor.c ?

Yes.

util/keyval.c provides keyval_parse() to parse QEMU's traditional
KEY=VALUE,... syntax into a QDict.

The really traditional part of that syntax is flat.  Dotted key
convention provides depth, and was grafted on later.

The big comment in keyval.c spells out the flaws^Wcompromises we had to
accept to bridge the gap between KEY=VALUE,... and JSON.

qobject_input_visitor_new_keyval() converts a QDict made with
keyval_parse() into a QAPI type.

>    c) What's the right way to build the output - is it to use the
>       existing qobject_output_visitor and add a qobject_to_keyval - or
>       is it a job for a new visitor?

For the input visitor, I chose the former, i.e. to do the keyval visitor
as a variant of the existing qobject input visitor, simply because the
two share so much code.

I've since found talking about the two variants awkward due to
difficulties naming them.  Perhaps coining distinct, pithy names for
the two would help.

I recommend to keep output and input visitors paired, at least at the
interface.