[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-block] [PULL 20/37] qcow2: Give the refcount cache the minimum pos
From: |
Kevin Wolf |
Subject: |
[Qemu-block] [PULL 20/37] qcow2: Give the refcount cache the minimum possible size by default |
Date: |
Tue, 15 May 2018 17:40:16 +0200 |
From: Alberto Garcia <address@hidden>
The L2 and refcount caches have default sizes that can be overridden
using the l2-cache-size and refcount-cache-size (an additional
parameter named cache-size sets the combined size of both caches).
Unless forced by one of the aforementioned parameters, QEMU will set
the unspecified sizes so that the L2 cache is 4 times larger than the
refcount cache.
This is based on the premise that the refcount metadata needs to be
only a fourth of the L2 metadata to cover the same amount of disk
space. This is incorrect for two reasons:
a) The amount of disk covered by an L2 table depends solely on the
cluster size, but in the case of a refcount block it depends on
the cluster size *and* the width of each refcount entry.
The 4/1 ratio is only valid with 16-bit entries (the default).
b) When we talk about disk space and L2 tables we are talking about
guest space (L2 tables map guest clusters to host clusters),
whereas refcount blocks are used for host clusters (including
L1/L2 tables and the refcount blocks themselves). On a fully
populated (and uncompressed) qcow2 file, image size > virtual size
so there are more refcount entries than L2 entries.
Problem (a) could be fixed by adjusting the algorithm to take into
account the refcount entry width. Problem (b) could be fixed by
increasing a bit the refcount cache size to account for the clusters
used for qcow2 metadata.
However this patch takes a completely different approach and instead
of keeping a ratio between both cache sizes it assigns as much as
possible to the L2 cache and the remainder to the refcount cache.
The reason is that L2 tables are used for every single I/O request
from the guest and the effect of increasing the cache is significant
and clearly measurable. Refcount blocks are however only used for
cluster allocation and internal snapshots and in practice are accessed
sequentially in most cases, so the effect of increasing the cache is
negligible (even when doing random writes from the guest).
So, make the refcount cache as small as possible unless the user
explicitly asks for a larger one.
Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>
---
block/qcow2.h | 4 ----
block/qcow2.c | 31 +++++++++++++++++++------------
tests/qemu-iotests/137.out | 2 +-
3 files changed, 20 insertions(+), 17 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index adf5c3950f..01b5250415 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -77,10 +77,6 @@
#define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */
#define DEFAULT_L2_CACHE_BYTE_SIZE 1048576 /* bytes */
-/* The refblock cache needs only a fourth of the L2 cache size to cover as many
- * clusters */
-#define DEFAULT_L2_REFCOUNT_SIZE_RATIO 4
-
#define DEFAULT_CLUSTER_SIZE 65536
diff --git a/block/qcow2.c b/block/qcow2.c
index 2f36e632f9..6d532470a8 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -802,23 +802,30 @@ static void read_cache_sizes(BlockDriverState *bs,
QemuOpts *opts,
} else if (refcount_cache_size_set) {
*l2_cache_size = combined_cache_size - *refcount_cache_size;
} else {
- *refcount_cache_size = combined_cache_size
- / (DEFAULT_L2_REFCOUNT_SIZE_RATIO + 1);
- *l2_cache_size = combined_cache_size - *refcount_cache_size;
+ uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE;
+ uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8);
+ uint64_t min_refcount_cache =
+ (uint64_t) MIN_REFCOUNT_CACHE_SIZE * s->cluster_size;
+
+ /* Assign as much memory as possible to the L2 cache, and
+ * use the remainder for the refcount cache */
+ if (combined_cache_size >= max_l2_cache + min_refcount_cache) {
+ *l2_cache_size = max_l2_cache;
+ *refcount_cache_size = combined_cache_size - *l2_cache_size;
+ } else {
+ *refcount_cache_size =
+ MIN(combined_cache_size, min_refcount_cache);
+ *l2_cache_size = combined_cache_size - *refcount_cache_size;
+ }
}
} else {
- if (!l2_cache_size_set && !refcount_cache_size_set) {
+ if (!l2_cache_size_set) {
*l2_cache_size = MAX(DEFAULT_L2_CACHE_BYTE_SIZE,
(uint64_t)DEFAULT_L2_CACHE_CLUSTERS
* s->cluster_size);
- *refcount_cache_size = *l2_cache_size
- / DEFAULT_L2_REFCOUNT_SIZE_RATIO;
- } else if (!l2_cache_size_set) {
- *l2_cache_size = *refcount_cache_size
- * DEFAULT_L2_REFCOUNT_SIZE_RATIO;
- } else if (!refcount_cache_size_set) {
- *refcount_cache_size = *l2_cache_size
- / DEFAULT_L2_REFCOUNT_SIZE_RATIO;
+ }
+ if (!refcount_cache_size_set) {
+ *refcount_cache_size = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size;
}
}
diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out
index e28e1eadba..96724a6c33 100644
--- a/tests/qemu-iotests/137.out
+++ b/tests/qemu-iotests/137.out
@@ -22,7 +22,7 @@ refcount-cache-size may not exceed cache-size
L2 cache size too big
L2 cache entry size must be a power of two between 512 and the cluster size
(65536)
L2 cache entry size must be a power of two between 512 and the cluster size
(65536)
-L2 cache size too big
+Refcount cache size too big
Conflicting values for qcow2 options 'overlap-check' ('constant') and
'overlap-check.template' ('all')
Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of
the following: none, constant, cached, all
Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of
the following: none, constant, cached, all
--
2.13.6
- [Qemu-block] [PULL 10/37] blockjob: expose error string via query, (continued)
- [Qemu-block] [PULL 10/37] blockjob: expose error string via query, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 15/37] blockjob: Introduce block_job_ratelimit_get_delay(), Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 11/37] blockjob: Fix assertion in block_job_finalize(), Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 16/37] blockjob: Add block_job_driver(), Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 14/37] blockjob: Implement block_job_set_speed() centrally, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 19/37] specs/qcow2: Clarify that compressed clusters have the COPIED bit reset, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 13/37] blockjob: Move RateLimit to BlockJob, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 12/37] blockjob: Wrappers for progress counter access, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 17/37] iotests: Split 214 off of 122, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 22/37] iotests: Add failure matching to common.qemu, Kevin Wolf, 2018/05/15
- [Qemu-block] [PULL 20/37] qcow2: Give the refcount cache the minimum possible size by default,
Kevin Wolf <=
[Qemu-block] [PULL 21/37] docs: Document the new default sizes of the qcow2 caches, Kevin Wolf, 2018/05/15
[Qemu-block] [PULL 26/37] block: Add BDRV_REQ_WRITE_UNCHANGED flag, Kevin Wolf, 2018/05/15
[Qemu-block] [PULL 28/37] block/quorum: Support BDRV_REQ_WRITE_UNCHANGED, Kevin Wolf, 2018/05/15
[Qemu-block] [PULL 18/37] Fix error message about compressed clusters with OFLAG_COPIED, Kevin Wolf, 2018/05/15
[Qemu-block] [PULL 23/37] iotests: Skip 181 and 201 without userfaultfd, Kevin Wolf, 2018/05/15