I also tested uploading to raw disk using qemu-nbd on my dev setup with
LIO storage over 1G nic.
For reference, blkdiscard --zeroout takes 38.5 seconds on this setup.
imageio example upload to /dev/nbd0[1]: 40.3 seconds
qemu-img convert to /dev/nbd0: 61.5 seconds
qemu-img convert to nbd+unix socket with unordered writes (-W): 87.3 seconds
qemu-img convert to nbd+unix socket: 96.1 seconds
There was a problem in this test, I used 10g LV but the image was 6g. imageio
write exactly what the client sent (6g), but qemu-img zero the entire device, which
is very slow with this storage.
I repeated these tests using 6g LV, and tested also -W with /dev/nbdX,
and nbdkit.
Here are corrected results:
imageio example upload to /dev/nbd2: 41.9 seconds
qemu-img convert to /dev/nbd2 with unordered writes (-W): 48.4 seconds
qemu-img convert to nbdkit unix socket with unordered writes (-W): 55.4 seconds
qemu-img convert to qenu-nbd unix socket with unordered writes (-W): 56.0 seconds
qemu-img convert to nbdkit unix socket: 56.3 seconds
qemu-img convert to /dev/nbd2: 60.6 seconds
qemu-img convert to qenu-nbd unix socket: 65.0 seconds
I zero the device using blkdiscard --zeroout before every run.
## using /dev/nbd2
# qemu-nbd -c /dev/nbd2 -f raw -t --cache=none --aio=native --discard=ignore --detect-zeroes=on /dev/27837a03-64f9-4f2b-abb0-daa2195b01ae/0b311496-c4c8-40c5-8725-f5c827f8b013
real 0m41.969s
user 0m0.275s
sys 0m0.542s
# time qemu-img convert -p -n -f raw -O raw -t none /var/tmp/fedora-27.img /dev/nbd2
(100.00/100%)
real 1m0.684s
user 0m0.320s
sys 0m0.907s
# time qemu-img convert -p -n -f raw -O raw -t none -W /var/tmp/fedora-27.img /dev/nbd2
(100.00/100%)
real 0m48.409s
user 0m0.307s
sys 0m0.995s
## qemu-nbd with nbd unix socket
# qemu-nbd -k /tmp/nbd.sock -f raw -t --cache=none --aio=native --discard=ignore --detect-zeroes=on /dev/27837a03-64f9-4f2b-abb0-daa2195b01ae/0b311496-c4c8-40c5-8725-f5c827f8b013
# time qemu-img convert -p -n -f raw -O raw -t none /var/tmp/fedora-27.img nbd+unix://?socket=/tmp/nbd.sock
(100.00/100%)
real 1m5.012s
user 0m0.157s
sys 0m0.644s
# time qemu-img convert -p -n -f raw -O raw -t none -W /var/tmp/fedora-27.img nbd+unix://?socket=/tmp/nbd.sock
(100.00/100%)
real 0m56.086s
user 0m0.163s
sys 0m0.624s
## nbdkit with unix socket
# src/nbdkit -f plugins/file/.libs/nbdkit-file-plugin.so file=/dev/27837a03-64f9-4f2b-abb0-daa2195b01ae/0b311496-c4c8-40c5-8725-f5c827f8b013 -U /tmp/nbd.sock
# time qemu-img convert -p -n -f raw -O raw -t none /var/tmp/fedora-27.img nbd+unix://?socket=/tmp/nbd.sock
(100.00/100%)
real 0m56.302s
user 0m0.145s
sys 0m0.611s
# time qemu-img convert -p -n -f raw -O raw -t none -W /var/tmp/fedora-27.img nbd+unix://?socket=/tmp/nbd.sock
(100.00/100%)
real 0m55.430s
user 0m0.122s
sys 0m0.615s