Re: [Qemu-devel] Re: Comparing New Image Formats: FVD vs. QED

[Chunqiang Tang]

> Am 15.02.2011 20:45, schrieb Chunqiang Tang:
> >> Chunqiang Tang/Watson/IBM wrote on 01/28/2011 05:13:27 PM:
> >> As you requested, I set up a wiki page for FVD at 
> > http://wiki.qemu.org/Features/FVD
> >> . It includes a summary of FVD, a detailed specification of FVD, and 
a 
> >> comparison of the design and performance of FVD and QED. 
> > 
> >> See the figure at http://wiki.qemu.org/Features/FVD/Compare . This 
> > figure 
> >> shows that the file creation throughput of NetApp's PostMark 
benchmark 
> > under 
> >> FVD is 74.9% to 215% higher than that under QED.
> > 
> > Hi Anthony,
> > 
> > Please let me know if more information is needed. I would appreciate 
your 
> > feedback and advice on the best way to proceed with FVD. 
> 
> Yet another file format with yet another implementation is definitely
> not what we need. We should probably take some of the ideas in FVD and
> consider them for qcow3.

I certainly agree that convergence is ideal, and FVD was designed from the 
very beginning for the purpose of convergence, by going a long way to make 
it flexible and fully configurable to suit different use cases. FVD is by 
no means another random format coming out of a random research hacking 
project. 

QCOW2 is feature rich (especially snapshots) and I am certain that it will 
be highly valuable to many users for a long foreseeable future, but it was 
simply not designed for achieving absolute high performance needed in 
server environments. One key thing missing in QEMU is a high-performance 
copy-on-write image format, which is dreadfully needed, at least based on 
my experience in Cloud. Even VirtualBox VDI is in a much better position 
in achieving high performance than QCOW2 because of VDI’s simple design. 
If we are going to move beyond QCOW2, as indicated by the move of adopting 
QED, regardless of the name of the next image format, be it QCOW3, QED, or 
FVD, it is important that we seriously learn the lessons from past image 
formats. Unfortunately, I was unaware of QED when it was under 
development. In my view, QED is so similar to QCOW2 (both by design and by 
implementation) that it does not achieve the goal of addressing the main 
limitations of QCOW2 and moving beyond to the next level. This will lead 
to further future image format fragmentation and suffering, rather than 
achieving the goal of convergence.

Why not make FVD the basis of QCOW3, with additional requirements 
incorporated? I have posted the spec and am quite open to suggestions. I 
performed a careful study on image formats, and truly believe that FVD is 
a great leap forward beyond any existing image formats (VMDK, VDI, VHD, 
QCOW2, etc.). FVD is quite mature as I spent more than half a year on 
hardening it. Even 5 years down the road, say by 2016, the chance of 
another image format beating the performance of FVD probably is slim. 

> However, I think some of them like the "no-alloc" mode aren't that
> useful: If I want the features and the performance of raw, I can just
> take raw.

FVD-no-alloc is a copy-on-write image format, whereas RAW is not. We are 
interested in copy-on-write image format. Otherwise, RAW would be great. 
The experiment uses a 50GB QCOW2/QED/FVD image based on a 1GB base image, 
which reflects the flavor of a configuration in Cloud. This info is 
available in the detailed experiment setup description.

> > BTW, I recently added QCOW2 into the performance comparison figure on 
> > wiki.
> 
> It's obvious why you have only one case for QED (it doesn't support
> anything else), but qcow2 works on block devices, too, and you can also
> use metadata preallocation. Are you aware of this?

The QCOW2 code shows that preallocation does not work with backing file. 
We need a copy-on-write image. Later I will add to the wiki new QCOW2 
results on block devices. My paper has results of old version QCOW2 on 
block devices. Based on my past much more extensive benchmarking of QCOW2 
in QEMU 0.12.3, increasing the metadata cache of QCOW2 would be more 
effective in increasing performance. I recommend increasing QCOW2's cache 
size to at least QED's level.