Re: [Qemu-devel] [PATCH] Introduce cache images for the QCOW2 format

[Eric Blake]

On 08/13/2013 11:03 AM, Kaveh Razavi wrote:
> Using copy-on-write images with the base image stored remotely is common
> practice in data centers. This saves significant network traffic by
> avoiding the transfer of the complete base image. However, the data
> blocks needed for a VM boot still need to be transfered to the node that
> runs the VM. On slower networks, this will create a bottleneck when
> booting many VMs simultaneously from a single VM image. Also,
> simultaneously booting VMs from more than one VM image creates a
> bottleneck at the storage device of the base image, if the storage
> device does not fair well with the random access pattern that happens

s/fair/fare/

> during booting.
> 
> This patch introduces a block-level caching mechanism by introducing a
> copy-on-read image that supports quota and goes in between the base
> image and copy-on-write image. This cache image can either be stored on
> the nodes that run VMs or on a storage device that can handle random
> access well (e.g. memory, SSD, etc.). This cache image is effective
> since usually only a very small part of the image is necessary for
> booting a VM. We measured 100MB to be enough for a default CentOS and
> Debian installations.
> 
> A cache image with a quota of 100MB can be created using these commands:
> 
> $ qemu-img create -f qcow2 -o
> cache_img_quota=104857600,backing_file=/path/to/base /path/to/cache
> $ qemu-img create -f qcow2 -o backing_file=/path/to/cache /path/to/cow

What is the QMP counterpart for hot-plugging a disk with the cache
attached?  Is this something that can integrate nicely with Kevin's
planned blockdev-add for 1.7?

> 
> The first time a VM boots from the copy-on-write image, the cache gets
> warm. Subsequent boots do not need to read from the base image.
> 
> The implementation is a small extension to the QCOW2 format. If you are
> interested to know more, please read this paper:
> http://cs.vu.nl/~kaveh/pubs/pdf/sc13.pdf

Please post this as a series, with patch 1 of the series being a doc
patch to docs/specs/qcow2.txt fully documenting this extension.  No one
else can be expected to interoperate with your extension if you don't
document it upstream.  I want to make sure that there are no races where
two competing processes both open a file read-write, and where the first
process requests to modify metadata (possibly by adding the cache
designation header, but also by making some other modification), but
where before that completes, the other process sees an incomplete
picture of the metadata.  We already document that qemu-img is liable to
misbehave, even when operating in read-only mode, on an image held
read-write by one qemu process; and I want to see in the formal docs
(without having to chase a link to the pdf) how you guarantee that this
is safe.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

signature.asc
Description: OpenPGP digital signature