Re: [Qemu-devel] [PATCH] Introduce cache images for the QCOW2 format

[Kaveh Razavi]

On 08/15/2013 10:32 AM, Stefan Hajnoczi wrote:
> I don't buy the argument about the page cache being evicted at any time:
>
> At the scale where caching is important, provisioning a measily 100 MB
> of RAM per guest should not be a challenge.
>
> cgroups can be used to isolate page cache between VMs if you want to
> guaranteed caches.
>
> But it could be more interesting not to isolate so that the page cache
> acts host-wide to reduce the overall I/O instead of narrowly focussing
> on caching 100 MB for a specific image even if it is rarely accessed.
>
> The real downside I see is that the page cache is volatile, so you could
> see heavy I/O if multiple hosts reboot at the same time.

At the VM hosts, the memory is mostly allocated to VMs. Without 
persisted caches, starting another VM from any of the possible backing 
VM images may or may not result in network traffic (depending on the 
page cache). Regardless of the page cache, the existing cache images 
persisted on the disk at hosts, can eliminate this at least on VM boot.

At the storage site however, I think it makes sense to dedicate memory 
for popular backing images (via tmpfs rather than page cache). The data 
blocks of the popular images used for booting will be accessed by all 
VMs starting from these "template" images.

> Streaming offers a rate limiting parameter so you can tune it to the
> network conditions.
>
> Copying the full image doesn't just reduce load on the NFS server, it
> also means guests can continue to run if the NFS server becomes
> unreachable.  That's an important property for reliability.

I am not really sure whether copying the entire image reduces the load 
on the NFS server, specially at scale. If copying the entire image at 
scale is desired/necessary, peer-to-peer approaches are documented to 
perform better. They are mostly implemented at the host file-system 
layer though (search for e.g. VMTorrent). I agree on the reliability 
consideration if you deal with an unreliable (remote) file-system.

> 1)
> It is persistent.  The backing file chain looks like this:
>
>    /nfs/template.qcow2 <- /local/cache.qcow2 <- /local/vm001.qcow2
>
> The cache is a regular qcow2 image file that is persistent.  The discard
> command is used to evict data from the file.  Copy-on-read accesses are
> used to populate the cache when the guest submits a read request.
>
> 2)
> You can set cache size or other parameters as a qemu-nbd option (this
> doesn't exist but could be implemented):
>
>    $ qemu-img create -f qcow2 -o backing_file=/nfs/template.qcow2 cache.qcow2
>    $ qemu-nbd --options cache-size=100MB,evict=lru cache.qcow2
>
> So it's the qemu-nbd process that performs the cache housekeeping work.
> The cache.qcow2 file itself just persists data and isn't aware of cache
> settings.

OK, this is better, since the user can also define a policy _and_ the 
cache can be shared by different VMs at the creation time without races. 
With an eviction policy 'none' in combination with cache_size, only the 
first accessed data blocks get cached, essentially providing the same 
functionality as this patch.

Kaveh

smime.p7s
Description: S/MIME Cryptographic Signature