Re: [Qemu-devel] [RFC][PATCH v3 00/24] instrument: Let the user wrap/override specific event tracing routines

[Stefan Hajnoczi]

On Wed, Apr 24, 2013 at 02:17:22PM +0200, Lluís Vilanova wrote:
> Stefan Hajnoczi writes:
> 
> > On Sun, Apr 21, 2013 at 09:11:30PM +0200, Lluís Vilanova wrote:
> >> TODO: Operations 'instr_load' and 'instr_unload' are not thread safe.
> >> (qemu_cpu_kick?)
> >> 
> >> TODO: Do cmdline actions have to be implemented on top of QMP routines?
> >> 
> >> TODO: HMP and QMP interfaces only accept one argument to "instr-load".
> >> 
> >> TODO: Replace programmatic 'InstrLoadError' in favour of QAPI's 
> >> 'InstrLoadCode'?
> >> (harder to find using code navigation tools, as it's auto-generated; but
> >> provides a single point to manage the enumeration values, which is less
> >> error-prone)
> >> 
> >> 
> >> The whole set of patch series is available at:
> >> https://projects.gso.ac.upc.edu/projects/qemu-dbi
> >> 
> >> Adds the "instrument" event property to declare which tracing events in 
> >> QEMU
> >> must be instrumentable. Still, in case the user only wants to wrap around 
> >> the
> >> tracing events, the original tracing implementation is accessible through 
> >> the
> >> appropriate routines.
> >> 
> >> The instrumentation can be performed either through a dynamically-loaded 
> >> library
> >> or through user-provided code that is compiled into QEMU itself (useful 
> >> when
> >> instrumenting high-frequency events, as the fast-path of the 
> >> instrumentation can
> >> be inlined into QEMU).
> >> 
> >> As a side-effect this series adds an API for the instrumentation code to 
> >> have
> >> some basic interaction with QEMU.
> >> 
> >> See the documentation added in the first patch for more information.
> >> 
> >> Signed-off-by: Lluís Vilanova <address@hidden>
> >> ---
> 
> > If I understand correctly this series allows trace events to be exported
> > as a shared library API.  It can be used with instrumentation libraries
> > (shared objects), which avoids rebuilding QEMU for each instrumentation
> > set.
> 
> > I'm skeptical of the effort required to do this (and maintain it) when
> > it's easy to keep several git branches - one for each instrumentation
> > set - and rebuild.
> 
> > Trace events are not an API.  They are not stable.  Therefore these
> > dynamic instrumentation libraries would be broken when QEMU changes.
> 
> > Maybe I don't understand the application well enough to see the benefit?
> 
> True, changing QEMU's tracing events will break the instrumentation libraries,
> and the user should be completely aware of that, given that the library is
> actually instrumenting specific tracing events (not just some well-established
> API). You can think of this as yet another tracing backend, with the ability 
> to
> interact with QEMU.
> 
> The actually interesting part is in the following series (which are also
> publicly available on the url of the cover), which adds some generic tracing
> events that should not change between versions, and can thus be safely 
> assumed.
> Such events are related to the guest code, including (but not limited to) 
> guest
> memory accesses or instruction execution. As we discussed in some very old try
> at this, what all of this provides is the ability to instrument guest code for
> all architectures supported by QEMU.
> 
> In fact, the ability to instrument all tracing events could be seen as just a
> side-effect of the implementation; but one that can come in handy in certain
> cases when developing ad-hoc trace analyses.
> 
> Does this make more sense now?

Thanks for explaining.

People doing this level of instrumentation probably need to do QEMU
hacking anyway for deep analysis, perhaps pausing and resuming the
guest, checking other guest RAM locations, grabbing device state, maybe
snapshotting the guest.

Is it worth building and maintain a stable API?  It means that folks
working on TCG, for example, do not have the freedom to change/delete
trace events.  They are now stuck maintaining trace event semantics no
matter how TCG will evolve in the future.

It would be nice to make TCG multi-threaded soon.  How will
instrumentation libraries handle this?  Existing ones probably won't,
they'll break unless QEMU restricts itself back to single-threaded mode.

Given that the group of users for this feature is so small while the
burden of supporting this is rather large, it seems like writing
instrumentation code straight in a QEMU source tree is the most powerful
and viable solution.

Basically I think putting a stable API in place here isn't going to fly.
In terms of the tracing subsystem I don't mind, but I think it's a
question for the larger QEMU community.

Stefan