Re: [Qemu-devel] Modularizing QEMU RFC

[Daniel P. Berrange]

On Fri, Jul 31, 2015 at 05:45:42PM +0200, Marc Marí wrote:
> Hi everyone
> 
> I propose improving the current modular driver system for QEMU so it
> can benefit everybody in speed and flexibility. I'm looking for other
> ideas, comments, critics, etc.
> 
> - Background -
> In order to speed up QEMU, I'm looking at the high number of libraries
> and dependencies that it loads. I've generated a QEMU image that needs
> 145 shared libraries to start, and there were still some options
> disabled. This is a lot, and this means that it is slow.
> 
> So I've been looking at actual module system. Yes, QEMU does have a
> module system, but disabled by default. The problem is, the modules get
> loaded always during startup. This means, booting with modules enabled
> is even slower, because loading at runtime is slower that letting the
> linker do all the work at the start. At this point, I doubt of the
> benefits of the actual modular system.
> 
> But, if disabling the actual block modules (iscsi, curl, rbd, gluster,
> ssh and dmg) gives 40 ms of speedup, I think is worth an effort of
> improving modules, and modularizing new parts
> 
> - Current module flow -
> The current module system is based on shared libraries. Each of these
> libraries has a constructor that registers the BlockDriver structure to
> the global bdrv_drivers list. This list is later searched by the type,
> the protocol, etc.
> 
> - Proposals -
> I have in mind two ideas, which are not mutually exclusive:

[snip]

> - My comments on my proposals -
> Ideally, I'd like a mixed solution. The user can specify what wants to
> load, but also, when something is not found, it is automatically
> searched.
> 
> In both options, the current module system has to be partly rewritten,
> and some of the current drivers with module capability might need to be
> modified to adapt to the new specifications.
> 
> And, a part from improving the current modular interface, there are a
> lot of other devices that might benefit from it, not just the block
> devices.
> 
> I still haven't looked at the memory footprint of QEMU, but I'm sure
> that the QEMU binary will lose a lot of weight with this addition.

One think I don't see mentioned is how this impacts on QEMU feature
detection by apps. For example, the recommended approach currnetly
is to launch QEMU with 'qemu-system-BLAH --machine none -qmp /some/sock'
and then query QMP for lists of devices supported, list of various
backends and other features.

If you're going to suggest a fully modular system, then when doing
QMP feature detection we still need to see the full list of features.
So either that implies all the metadata associated with the modules
remains built-in to QEMU, so QMP can answer this without lodaing the
modules, or the QMP feature detection must imply auto-loading of all
modules that exist.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|