[Qemu-devel] Re: libvirt vs. in-qemu management

[Daniel P. Berrange]

On Mon, Apr 05, 2010 at 11:11:48PM +0200, Alexander Graf wrote:
> Howdy,
> 
> I've been thinking a bit further on the whole issue around 
> libvirt and why the situation as is isn't satisfying. I came
> to the following points that currently hurt building ease of
> use for KVM:
> 
> 1) Brand
> 
> This is one of the major issues we have ourselves when it comes
> to appliances. We can ship appliances built for VMware. We can 
> ship appliances built for Xen. But we can't ship appliances 
> built for KVM, because there is no single management app we could
> target. That destroys the KVM brand IMHO.

With appliances there are two core aspects

 1. The description of VM hardware requirements
 2. The disk format

Traditionally VMware appliances have shipped a VMX file for 1. and
a VMDK file for 2. 

Shipping the native config file format with an appliance though is
the wrong thing to be doing. The native config format describes the
configuration for a VM for a specific deployment. This is not the
same as describing the hardware requirements of an appliance. As
the most simple example, a native config would have hardcoded disk
paths, or a specific choice of host network connectivity. Neither
of these things have any business being in the appliance config.

For this reason, there are now specific appliance formats. Libvirt
has long had its own appliance format (virt-image) which is separate
from the main XML format, so it avoids hardcoding deployment specific
options. There is also the vendor neutral OVF format which is widely 
supported by many mgmt tools. 

If people want to ship QEMU appliances I don't think libvirt is 
causing any problems here. Simply ship a OVF description + either
raw or qcow2 disk image. Any app, libvirt, or not could work with
that.

> 2) Machine description
> 
> If we build an appliance, we also create a configuration file that
>  describes the VM. We can create .vmx files, we can create xen config
> files. We can not create KVM config files. There are none. We could
> create shell scripts, but would that help?

As described above, appliances really don't want to be using the 
native configuration formats, they want a higher level format like
OVF. The only reason soo many people ship .vmx files is that this
predates the OVF format's existance.

With qdev you can load most options from a config file using the new
'-readconfig <file>' arg, but i guess there's more to be included
there still. 

> 3) Configuration conversion
> 
> Party due to qemu not having a configuration format, partly due to 
> libvirt's ambivalent approach, there is always conversion in 
> configuration formats involved. I think this is the main reason for
> the feature lag. If there wasn't a conversion step, there wouldn't 
> be lag. You could just hand edit the config file and be good.

[snip]

> Point 3 is the really tough one. It's the very basis of libvirt. And 
> it's plain wrong IMHO. I hate XML. I hate duplicated efforts. The 
> current conversion involves both. Every option added to qemu needs to 
> be added to libvirt. In XML. Bleks.

In the previous thread on this topic, I've already stated that we're
interested in providing a means to pass QEMU config options from
libvirt prior to their full modelling in the XML, to reduce, or completely 
eliminate any time-lag in using new features.


> Reading on IRC I seem to not be the only person thinking that, just 
> the first one mentioning this aloud I suppose. But that whole XML mess
> really hurts us too. Nobody wants to edit XML files. Nobody wants to 
> have two separate syntaxes to describe the same thing. It complicates
> everything without a clear benefit. And it puts me in a position where 
> I can't help people because I don't know the XML format. That should 
> never happen.

Even if an app was using QEMU directly, you can't presume that the app 
will use QEMU's config file as its native format. Many apps will store
configs in their own custom format (or in a database) and simply generate
the QEMU config data on the fly when starting a VM. In the same way libvirt
will  generate QEMU config data on the fly when starting a VM. Having many
config formats & conversion / generation of the fly is a fact of life no 
matter what mgmt system you use.

The key point is that it needs to be really easy to get at the generated
QEMU config data to see what is actually being run. libvirt will always
save the exact QEMU config data it generates into a log file for this
exact purpose (/var/log/libvirt/qemu/$VMNAME.log).  The complexity here
is that you can't directly run this if TAP devices are in use for the
networking since it'll expect TAP FDs to be passed down. Although you 
could allow QEMU to open the TAP devices, this is not good  for security 
separation of QEMU from the host OS, so I'm not sure it can be easily
avoided. 

One attempt to make life easier, was that we added a libvirt command 
'virsh domxml-to-native' command which given a libvirt XML config file
would return a set of QEMU command line args which *can* be used to
start the VM directly. This is basically identical to the normal QEMU
args libvirt generates but relies on the ifup script to create TAP
networking instead.

Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|