[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [5323] Implement an fd pool to get real AIO with posix-
From: |
Ryan Harper |
Subject: |
Re: [Qemu-devel] [5323] Implement an fd pool to get real AIO with posix-aio |
Date: |
Fri, 26 Sep 2008 12:59:27 -0500 |
User-agent: |
Mutt/1.5.6+20040907i |
* Anthony Liguori <address@hidden> [2008-09-26 11:03]:
> Revision: 5323
> http://svn.sv.gnu.org/viewvc/?view=rev&root=qemu&revision=5323
> Author: aliguori
> Date: 2008-09-26 15:59:29 +0000 (Fri, 26 Sep 2008)
>
> Log Message:
> -----------
> Implement an fd pool to get real AIO with posix-aio
>
> This patch implements a simple fd pool to allow many AIO requests with
> posix-aio. The result is significantly improved performance (identical to
> that
> reported for linux-aio) for both cache=on and cache=off.
>
> The fundamental problem with posix-aio is that it limits itself to one thread
> per-file descriptor. I don't know why this is, but this patch provides a
> simple
> mechanism to work around this (duplicating the file descriptor).
>
> This isn't a great solution, but it seems like a reasonable intermediate step
> between posix-aio and a custom thread-pool to replace it.
>
> Ryan Harper will be posting some performance analysis he did comparing
> posix-aio
> with fd pooling against linux-aio. The size of the posix-aio thread pool and
> the fd pool were largely determined by him based on this analysis.
I'll have some more data to post in a bit, but for now, bumping the fd
pool up to 64 and ensuring we init aio to support a thread per fd, we
mostly match linux aio performance with a simpler implementation. For
randomwrites, fd_pool lags a bit, but I've got other data that shows in
most scenarios, fd_pool matches linux aio performance and does so with
less CPU consumption.
Results:
16k randwrite 1 thread, 74 iodepth | MB/s | avg sub lat (us) | avg comp lat (ms)
-----------------------------------+------+------------------+------------------
baremetal (O_DIRECT, aka cache=off)| 61.2 | 13.07 | 19.59
kvm: cache=off posix-aio w/o patch | 4.7 | 3467.44 | 254.08
kvm: cache=off linux-aio | 61.1 | 75.35 | 19.57
kvm: cache=on posix-aio w/o patch |127.0 | 115.78 | 9.19
kvm: cache=on posix-aio w/ patch |126.0 | 67.35 | 9.30
------------ new results ----------+------+------------------+------------------
kvm:cache=off posix-aio fd_pool[16]| 33.5 | 14.28 | 49.19
kvm:cache=off posix-aio fd_pool[64]| 51.1 | 14.86 | 23.66
16k write 1 thread, 74 iodepth | MB/s | avg sub lat (us) | avg comp lat (ms)
-----------------------------------+------+------------------+------------------
baremetal (O_DIRECT, aka cache=off)|128.1 | 10.90 | 9.45
kvm: cache=off posix-aio w/o patch | 5.1 | 3152.00 | 231.06
kvm: cache=off linux-aio |130.0 | 83.83 | 8.99
kvm: cache=on posix-aio w/o patch |184.0 | 80.46 | 6.35
kvm: cache=on posix-aio w/ patch |165.0 | 70.90 | 7.09
------------ new results ----------+------+------------------+------------------
kvm:cache=off posix-aio fd_pool[16]| 78.2 | 58.24 | 15.43
kvm:cache=off posix-aio fd_pool[64]|129.0 | 71.62 | 9.11
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
address@hidden