Re: [Qemu-devel] Re: Strategic decision: COW format

[Kevin Wolf]

Am 22.02.2011 09:37, schrieb Markus Armbruster:
> Anthony Liguori <address@hidden> writes:
> 
>> On 02/18/2011 03:57 AM, Kevin Wolf wrote:
>>> Am 18.02.2011 10:12, schrieb Markus Armbruster:
>>>    
>>>> Kevin Wolf<address@hidden>  writes:
>>>>
>>>>      
>>>>> Am 15.02.2011 20:45, schrieb Chunqiang Tang:
>>>>>        
>>>>>>> Chunqiang Tang/Watson/IBM wrote on 01/28/2011 05:13:27 PM:
>>>>>>> As you requested, I set up a wiki page for FVD at
>>>>>>>            
>>>>>> http://wiki.qemu.org/Features/FVD
>>>>>>          
>>>>>>> . It includes a summary of FVD, a detailed specification of FVD, and a
>>>>>>> comparison of the design and performance of FVD and QED.
>>>>>>>            
>>>>>>          
>>>>>>> See the figure at http://wiki.qemu.org/Features/FVD/Compare . This
>>>>>>>            
>>>>>> figure
>>>>>>          
>>>>>>> shows that the file creation throughput of NetApp's PostMark benchmark
>>>>>>>            
>>>>>> under
>>>>>>          
>>>>>>> FVD is 74.9% to 215% higher than that under QED.
>>>>>>>            
>>>>>> Hi Anthony,
>>>>>>
>>>>>> Please let me know if more information is needed. I would appreciate your
>>>>>> feedback and advice on the best way to proceed with FVD.
>>>>>>          
>>>>> Yet another file format with yet another implementation is definitely
>>>>> not what we need. We should probably take some of the ideas in FVD and
>>>>> consider them for qcow3.
>>>>>        
>>>> Got an assumption there: that the one COW format we need must be qcow3,
>>>> i.e. an evolution of qcow2.  Needs to be justified.  If that discussion
>>>> has happened on the list already, I missed it.  If not, it's overdue,
>>>> and then we better start it right away.
>>>>      
>>> Right. I probably wasn't very clear about what I mean with qcow3 either,
>>> so let me try to summarize my reasoning.
>>>
>>>
>>> The first point is an assumption that you made, too: That we want to
>>> have only one format. I hope it's easy to agree on this, duplication is
>>> bad and every additional format creates new maintenance burden,
>>> especially if we're taking it serious. Until now, there were exactly two
>>> formats for which we managed to do this, raw and qcow2. raw is more or
>>> less for free, so with the introduction of another format, we basically
>>> double the supported block driver code overnight (while not doubling the
>>> number of developers).
>>>    
>>
>> Not sure what project you're following, but we've had an awful lot of
>> formats before qcow2 :-)
>>
>> And qcow2 was never all that special, it just was dropped in the code
>> base one day.  You've put a lot of work into qcow2, but there are
>> other folks that are contributing additional formats and that means
>> more developers.
>>
>>> The consequence of having only one file format is that it must be able
>>> to obsolete the existing ones, most notably qcow2. We can only neglect
>>> qcow1 today because we can tell users to use qcow2. It supports
>>> everything that qcow1 supports and more. We couldn't have done this if
>>> qcow2 lacked features compared to qcow1.
>>>
>>> So the one really essential requirement that I see is that we provide a
>>> way forward for _all_ users by maintaining all of qcow2's features. This
>>> is the only way of getting people to not stay with qcow2.
>>>
>>>
>>> Of course, you could invent another format that implements the same
>>> features, but I think just carefully extending qcow2 has some real
>>> advantages.
>>>
>>> The first is that conversion of existing images would be really easy.
>>> Basically increment the version number in the header file and you're
>>> done. Structures would be compatible.
>>
>> qemu-img convert is a reasonable path for conversion.
>>
>>>   If you compare it to file systems,
>>> I rarely ever change the file system on a non-empty partition. Even if I
>>> wanted, it's usually just too painful. Except when I was able to use
>>> "tune2fs -j" to make ext3 out of ext2, that was really easy. We can
>>> provide the same for qcow2 to qcow3 conversion, but not with a
>>> completely new format.
>>>
>>> Also, while obsoleting a file format means that we need not put much
>>> effort in its maintenance, we still need to keep the code around for
>>> reading old images. With an extension of qcow2, it would be the same
>>> code that is used for both versions.
>>>
>>> Third, qcow2 already exists, is used in practice and we have put quite
>>> some effort into QA. At least initially confidence would be higher than
>>> in a completely new, yet untested format. Remember that with qcow3 I'm
>>> not talking about rewriting everything, it's a careful evolution, mostly
>>> with optional additions here and there.
>>>    
>>
>> My requirements for a new format are as followed:
>>
>> 1) documented, thought-out specification that is covered under and
>> open license with a clear process for extension.
>>
>> 2) ability to add both compatible and incompatible features in a
>> graceful way
>>
>> 3) ability to achieve performance that's close to raw.  I want our new
>> format to be able to be used universally both for servers and
>> desktops.
> 
> I'd like to add
> 
> 4) minimize complexity and maximize maintainability of the code.  I'd
> gladly sacrifice nice-to-have features for that.

Especially if they are features that only other users use, right?

What's the "Sankt-Florians-Prinzip" called in English?

>> I think qcow2 has some misfeatures like compression and internal
>> snapshots.  I think preserving those misfeatures is a mistake because
>> I don't think we can satisfy the above while trying to preserve those
>> features.  If the image format degrades when those features are
>> enabled, then it decreases confidence in the format.
> 
> I'm inclined to agree.  There's one way to prove us wrong: implement the
> misfeatures without compromising the requirements.

*sigh*

It starts to get annoying, but if you really insist, I can repeat it
once more: These features that you don't need (this is the correct
description for what you call "misfeatures") _are_ implemented in a way
that they don't impact the "normal" case. And they are it today.

Kevin