Re: [Qemu-devel] [PATCH 1/5] RFC: Efficient VM backup for qemu (v1)

[Kevin Wolf]

Am 21.11.2012 14:25, schrieb Dietmar Maurer:
>>> AFAIK qcow2 file cannot store data out of order. In general, an backup
>>> fd is not seekable, and we only want to do sequential writes. Image format
>> always requires seekable fds?
>>
>> Ah, this is what you mean by "out of order". Just out of curiosity, what are
>> these non-seekable backup fds usually?
> 
> /dev/nst0 ;-)

Sure. :-)

> But there are better examples. Usually you want to use some kind of
> compression, and you do that with existing tools:
> 
> # backup to stdout|gzip|...

When you use an image/archive format anyway, you could use a compression
mechanism that it already supports.

> A common usage scenario is to pipe a backup into a restore (copy)
> 
> # backup to stdout|ssh to remote host -c 'restore from stdin'

This is a good one. I believe our usual solution would have been to
backup to a NBD server on the remote host instead.

In general I can see that being able to pipe it to other programs could
be nice. I'm not sure if it's an absolute requirement. Would your tools
for taking the backup employ any specific use of pipes?

> It is also a performance question. Seeks are terrible slow.

You wouldn't do it a lot. Only for metadata, and you would only write
out the metadata once the in-memory cache is full.

>> In principle even for this qcow2 could be used as an image format, however
>> the existing implementation wouldn't be of much use for you, so it loses
>> quite a bit of its attractiveness.
>>
>>> Anyways, a qcow2 file is really complex beast - I am quite unsure if I
>>> would use that for backup if it is possible.
>>>
>>> That would require any external tool to include >=50000 LOC
>>>
>>> The vma reader code is about 700 LOC (quite easy).
>>
>> So what? qemu-img is already there.
> 
> Anyways, you already pointed out that the existing implementation does not 
> work.

I'm still trying to figure out the real requirements to think some more
about it. :-)

> But I already expected such discussion. So maybe it is better we simply pipe 
> all data to an external binary?
> We just need to define a minimal protocol. 
> 
> In future we can produce different archivers as independent/external binaries?

You shouldn't look at discussions as a bad thing. We're not trying to
block your changes, but to understand and possibly improve them.

Yes, discussions mean that it takes a bit longer to get things merged,
but they also mean that usually something better is merged in the end
that actually fits well in qemu's design, is maintainable, generic and
so on. Evading the discussions by keeping code externally wouldn't
improve things.

Which doesn't mean that external archivers are completely out of the
question, but I would only consider them if there's a good technical
reason to do so.

So if eventually we come to the conclusion that vma (or for that matter,
anything else in your patches) is the right solution, let's take it. But
first please give us the chance to understand the reasons of why you did
things the way you did them, and to discuss the pros and cons of
alternative solutions.

Kevin