[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-devel] [Bug 595117] Re: qemu-nbd slow and missing "writeback" cach

From: Serge Hallyn
Subject: [Qemu-devel] [Bug 595117] Re: qemu-nbd slow and missing "writeback" cache option
Date: Mon, 13 Dec 2010 14:24:14 -0000


did upstream ever accept your patch?

You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

  qemu-nbd slow and missing "writeback" cache option

Status in QEMU:
Status in “qemu-kvm” package in Ubuntu:

Bug description:
  Binary package hint: qemu-kvm

dpkg -l | grep qemu
ii  kvm                                  
1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9            dummy transitional 
pacakge from kvm to qemu-
ii  qemu                                 0.12.3+noroms-0ubuntu9                 
               dummy transitional pacakge from qemu to qemu
ii  qemu-common                          0.12.3+noroms-0ubuntu9                 
               qemu common functionality (bios, documentati
ii  qemu-kvm                             0.12.3+noroms-0ubuntu9                 
               Full virtualization on i386 and amd64 hardwa
ii  qemu-kvm-extras                      0.12.3+noroms-0ubuntu9                 
               fast processor emulator binaries for non-x86
ii  qemu-launcher                        1.7.4-1ubuntu2                         
               GTK+ front-end to QEMU computer emulator
ii  qemuctl                              0.2-2                                  
               controlling GUI for qemu

lucid amd64.

qemu-nbd is a lot slower when writing to disk than say nbd-server.

It appears it is because by default the disk image it serves is open with 
O_SYNC. The --nocache option, unintuitively, makes matters a bit better because 
it causes the image to be open with O_DIRECT instead of O_SYNC.

The qemu code allows an image to be open without any of those flags, but 
unfortunately qemu-nbd doesn't have the option to do that (qemu doesn't allow 
the image to be open with both O_SYNC and O_DIRECT though).

The default of qemu-img (of using O_SYNC) is not very sensible because anyway, 
the client (the kernel) uses caches (write-back), (and "qemu-nbd -d" doesn't 
flush those by the way). So if for instance qemu-nbd is killed, regardless of 
whether qemu-nbd uses O_SYNC, O_DIRECT or not, the data in the image will not 
be consistent anyway, unless "syncs" are done by the client (like fsync on the 
nbd device or sync mount option), and with qemu-nbd's O_SYNC mode, those 
"sync"s will be extremely slow.

Attached is a patch that adds a --cache={off,none,writethrough,writeback} 
option to qemu-nbd.

--cache=off is the same as --nocache (that is use O_DIRECT), writethrough is 
using O_SYNC and is still the default so this patch doesn't change the 
functionality. writeback is none of those flags, so is the addition of this 
patch. The patch also does an fsync upon "qemu-nbd -d" to make sure data is 
flushed to the image before removing the nbd.

Consider this test scenario:

dd bs=1M count=100 of=a < /dev/null
qemu-nbd --cache=<x> -c /dev/nbd0 a
cp /dev/zero /dev/nbd0
time perl -MIO::Handle -e 'STDOUT->sync or die$!' 1<> /dev/nbd0

With cache=writethrough (the default), it takes over 10 minutes to write those 
100MB worth of zeroes. Running a strace, we see the recvfrom and sentos delayed 
by each 1kb write(2)s to disk (10 to 30 ms per write).

With cache=off, it takes about 30 seconds.

With cache=writeback, it takes about 3 seconds, which is similar to the 
performance you get with nbd-server

Note that the cp command runs instantly as the data is buffered by the client 
(the kernel), and not sent to qemu-nbd until the fsync(2) is called.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]