[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [RFC] Disk integrity in QEMU

From: Anthony Liguori
Subject: Re: [Qemu-devel] [RFC] Disk integrity in QEMU
Date: Sat, 11 Oct 2008 15:35:09 -0500
User-agent: Thunderbird (X11/20080925)

Mark Wagner wrote:
Avi Kivity wrote:

I think one of the main things to be considered is the integrity of the
actual system call.  The Linux manpage for open() states the following
about the use of the O_DIRECT flag:

O_DIRECT (Since Linux 2.6.10)
Try to minimize cache effects of the I/O to and from this file.  In
general this will degrade performance, but it is useful  in  special
situations,  such as when applications do their own caching.  File
I/O is done directly to/from user space buffers.  The I/O is
synchronous, that is, at the completion of a read(2) or write(2),
data is guaranteed to  have  been  transferred.   Under  Linux  2.4
transfer  sizes, and the alignment of user buffer and file offset
must all be multiples of the logical block size of the file system.
Under Linux 2.6 alignment to 512-byte boundaries suffices.

If I focus on the sentence "The I/O is synchronous, that is, at
the completion of a read(2) or write(2), data is guaranteed to have
been transferred. ",

It's extremely important to understand what the guarantee is. The guarantee is that upon completion on write(), the data will have been reported as written by the underlying storage subsystem. This does *not* mean that the data is on disk.

If you have a normal laptop, your disk has a cache. That cache does not have a battery backup. Under normal operations, the cache is acting in write-back mode and when you do a write, the disk will report the write as completed even though it is not actually on disk. If you really care about the data being on disk, you have to either use a disk with a battery backed cache (much more expensive) or enable write-through caching (will significantly reduce performance).

In the case of KVM, even using write-back caching with the host page cache, we are still honoring the guarantee of O_DIRECT. We just have another level of caching that happens to be write-back.

I think there a bug here. If I open a
file with the O_DIRECT flag and the host reports back to me that
the transfer has completed when in fact its still in the host cache,
its a bug as it violates the open()/write() call and there is no
guarantee that the data will actually be written.

This is very important, O_DIRECT does *not* guarantee that data actually resides on disk. There are many possibly places that it can be cached (in the storage controller, in the disks themselves, in a RAID controller).

So I guess the real issue isn't what the default should be (although
the performance team at Red Hat would vote for cache=off),

The consensus so far has been that we want to still use the host page cache but use it in write-through mode. This would mean that the guest would only see data completion when the host's storage subsystem reports the write as having completed. This is not the same as cache=off but I think gives the real effect that is desired.

Do you have another argument for using cache=off?


Anthony Liguori

the real
issue is that we need to honor the system call from the guest. If
the file is opened with O_DIRECT on the guest, then the host needs
to honor that and do the same.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]