[Qemu-devel] Re: Notes on block I/O data integrity

[Christoph Hellwig]

On Tue, Aug 25, 2009 at 10:25:08PM +0200, Nikola Ciprich wrote:
> Hello Christopher,
> 
> thanks a lot vor this overview, it answers a lot of my questions!
> May I suggest You put it somewhere on the wiki so it doesn't get 
> forgotten in the maillist only?

I'll rather try to get the worst issues fixed ASAP.

> It also rises few new questions though. We have experienced postgresql
> database corruptions lately, two times to be exact. First time, I blamed
> server crash, but lately (freshly created) database got corrupted for the 
> second time and there were no crashes since the initialisation. The server
> hardware is surely OK. I didn't have much time to look into this
> yet, but Your mail just poked me to return to the subject. The situation
> is a bit more complex, as there are additional two layers of storage there:
> we're using SATA/SAS drives, network-mirrored by DRBD, clustered LVM on top
> of those, and finally qemu-kvm using virtio on top of created logical
> volumes. So there are plenty of possible culprits, but Your mention of virtio
> unsafeness while using cache=writethrough (which is the default for drive 
> types other then qcow) leads me to suspicion that this might be the reason of 
> the problem. Databases are sensitive for requests reordering, so I guess
> using virtio for postgres storage was quite stupid from me :(
> So my question is, could You please advise me a bit on the storage
> configuration? virtio performed much better then SCSI, but of course
> data integrity is crucial, so would You suggest rather using SCSI?
> DRBD doesn't have problem with barriers, clustered LVM SHOULD not 
> have problems with it, as we're using just striped volumes, but I'll
> check it to be sure. So is it safe for me to keep cache=writethrough
> for the database volume?

I'm pretty sure one of the many laters in your setup will not pass
through write barriers, so defintively make sure your write caches are
disabled.  Also right now virtio is not a good idea for data integrity.
The guest side fix for a setup with cache=writethrough or cache=none
on block device without volatile disk write cache is however a trivial
one line patch I've already submitted.  I've attached it below for
reference:

virtio-blk-drain
Description: Text document