[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-block] [Qemu-devel] drive-backup

From: John Snow
Subject: Re: [Qemu-block] [Qemu-devel] drive-backup
Date: Thu, 25 Feb 2016 14:53:36 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0

On 02/25/2016 02:49 AM, Stefan Priebe - Profihost AG wrote:
> Am 22.02.2016 um 23:08 schrieb John Snow:
>> On 02/22/2016 03:21 PM, Stefan Priebe wrote:
>>> Hello,
>>> is there any chance or hack to work with a bigger cluster size for the
>>> drive backup job?
>>> See:
>>> http://git.qemu.org/?p=qemu.git;a=blob;f=block/backup.c;h=16105d40b193be9bb40346027bdf58e62b956a96;hb=98d2c6f2cd80afaa2dc10091f5e35a97c181e4f5
>>> This is very slow with ceph - may be due to the 64k block size. I would
>>> like to check whether this is faster with cephs native block size of 4mb.
>>> Greets,
>>> Stefan
>> It's hardcoded to 64K at the moment, but I am checking in a patch to
>> round up the cluster size to be the bigger of (64k,
>> $target_cluster_size) in order to make sure that incremental backups in
>> particular never copy a fraction of a cluster. As a side-effect, the
>> same round-up will happen for all modes (sync=top,none,full).
>> If QEMU is aware of the target cluster size of 4MB, this would
>> immediately jump the copy-size up to 4MB clusters for you.
>> See: https://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg02839.html
> Thanks for your patches and thanks for your great answer. But our
> problem is not the target but the source ;-) The target has a local
> cache and don't care about the cluster size but the source does not.
> But it works fine if we change the default cluster size to 4MB. So it
> has point us to the right direction.
> Stefan

Ah, sorry, I misunderstood.

It's easy to change anyway! I am in favor of adding a configurable
parameter, as long as it keeps the other constraints I mentioned in mind.


>> Otherwise, after my trivial fix, you should find cluster_size to be a
>> mutable concept and perhaps one that you could introduce a runtime
>> parameter for if you could convince the necessary parties that it's
>> needed in the API.
>> You'd have to be careful in the case of incremental that all the various
>> cluster sizes work well together:
>> - Granularity of bitmap (Defaults to cluster size of source, or 64K if
>> unknown. can be configured to be arbitrarily larger.)
>> - Cluster size of source file (For qcow2, defaults to 64k)
>> - Cluster size of target file
>> - Cluster size of backup routine (Currently always 64K)
>> I think that LCM(source_cluster_size, target_cluster_size,
>> backup_cluster_size) = MAX(A, B, C) will always be a safe minimum.
>> Bitmap granularity is more flexible, and it is more efficient when the
>> bitmap granularity matches the backup cluster size, but it can cope with
>> mismatches. If the bitmap is smaller or larger than the backup cluster
>> size, it generally means that more data that is clean is going to be
>> transferred across the pipe.
>> (Hmm, I wonder if it's worth checking in code to adjust a bitmap
>> granularity after it has been created so people can easily experiment
>> with performance tweaking here.)
>> Let me know if any of my ramble sounds interesting :)
>> --John


reply via email to

[Prev in Thread] Current Thread [Next in Thread]