Re: [Qemu-devel] Re: KVM call agenda for Jan 25

[Stefan Hajnoczi]

On Tue, Mar 15, 2011 at 10:27 AM, Kevin Wolf <address@hidden> wrote:
> Am 14.03.2011 16:13, schrieb Dushyant Bansal:
>>>
>>> Nice that qemu-img convert isn't that far out by default on raw :).
>>>
>>> About Google Summer of Code, I have posted my take on applying and
>>> want to share that with you and qemu-devel:
>>>
>>> http://blog.vmsplice.net/2011/03/advice-for-students-applying-to-google.html
>>>
>> Thanks for sharing your experiences.
>>
>> After reading about qcow2 and qed and how they organize data (thanks to
>> the newly added qcow2 doc and discussions on the mailing list), this is
>> what I understand.
>>
>> So, the main difference between qed and qcow2 is the absence of
>> reference count structure in qed(means less meta data).
>> It improves performance due to:
>> 1. For write operations, less or no metadata to update.
>> 2. Data write and metadata write can be in any order
>>
>> This also means these features are no longer supported:
>> 1. Internal snapshots,
>> 2. CPU/device state snapshots,
>> 3. Compression,
>> 4. Encryption
>>
>> Now, coming to qed<-->qcow2 conversion, I want to clarify some things.
>>
>> 1. header_size: variable in qed, equals to cluster size in qcow2:
>> When will it be larger than 1 cluster in qed? So, what will happen to
>> that extra data on qed->qcow2 conversion.
>
> If you have an feature that is used in the original image, but cannot be
> represented in the new format, I think you should just get an error.
>
>> 2. L2 table size: equals to L1 table size in qed, equals to cluster size
>> in qcow2:
>> we need to take it into account during conversion.
>
> Right. I think we'll have to rewrite all of the metadata.
>
> I wonder if we can manage to have a nice block driver interface for
> in-place image conversions so that we don't only get a qed<->qcow2
> converter, but also can implement the interface in e.g. VMDK and get
> VMDK<->qcow2 and VMDK<->qed as well.

I think this will be tricky but would be very interested if someone
has ideas.  Code-wise an in-place converter probably needs access to
both format's on-disk structures or internal functions.  I don't think
abstracting this is easy because the more you abstract the less
control you have over keeping things in-place and cleanly putting the
new structures in place.

On the other hand, I think the starting point for a generic in-place
converter would be a loop that does something like bdrv_is_allocated()
but translates the guest position in the block device into an offset
into the image file.  That, together with some sort of free map or
space allocation bitmap would allow a generic approach to figuring out
the data mapping and which parts of the file can be safely used.

The big benefit doing an interface for in-place conversion is that we
can support n-to-n conversions with at most n converter code rather
than having to code n * n - n different in-place converters.

>> 3. refcount table:
>> qcow2->qed:we do not keep refcount structure
>> qed->qcow2: initialize refcount structure
>
> Yes, refcounts can be rebuilt after the mapping has been converted.
>
>> So, a qcow2->qed->qcow2 conversion means if earlier, qcow2 image was
>> using additional features{1-4}, all information related to that will be
>> lost.
>
> We shouldn't lose anything but just abort if a conversion isn't
> possible. The user can still use qemu-img convert for the more
> complicated cases, for example for removing encryption or compression.

I agree with Kevin on the points made.

Stefan