[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [5323] Implement an fd pool to get real AIO with posix-
From: |
Ryan Harper |
Subject: |
Re: [Qemu-devel] [5323] Implement an fd pool to get real AIO with posix-aio |
Date: |
Fri, 26 Sep 2008 13:50:57 -0500 |
User-agent: |
Mutt/1.5.6+20040907i |
* Anthony Liguori <address@hidden> [2008-09-26 13:37]:
> Ryan Harper wrote:
> >* Anthony Liguori <address@hidden> [2008-09-26 11:03]:
> >kvm: cache=on posix-aio w/o patch |127.0 | 115.78 | 9.19
> >kvm: cache=on posix-aio w/ patch |126.0 | 67.35 | 9.30
> >
>
> It looks like 127mb/s is pretty close to the optimal cached write time.
> When using caching, writes can complete almost immediately so it's not
> surprising that submission latency is so low (even though it's blocking
> during submission).
>
> I am surprised that w/patch has a latency that's so high. I think that
> suggests that requests are queuing up. I bet increasing the aio_num
> field would reduce this number.
Yeah, there is plenty of room to twiddle with the threads and number of
outstanding ios, but that'll take quite a bit of time to generate the
data and compare.
> >------------ new results
> >----------+------+------------------+------------------
> >kvm:cache=off posix-aio fd_pool[16]| 33.5 | 14.28 | 49.19
> >kvm:cache=off posix-aio fd_pool[64]| 51.1 | 14.86 | 23.66
> >
>
> I assume you tried to bump from 64 to something higher and couldn't make
> up the lost bandwidth?
Very slightly, switching to 128 threads/fds gave another 1MB/s.
> >16k write 1 thread, 74 iodepth | MB/s | avg sub lat (us) | avg comp
> >lat (ms)
> >-----------------------------------+------+------------------+------------------
> >baremetal (O_DIRECT, aka cache=off)|128.1 | 10.90 | 9.45
> >kvm: cache=off posix-aio w/o patch | 5.1 | 3152.00 | 231.06
> >kvm: cache=off linux-aio |130.0 | 83.83 | 8.99
> >kvm: cache=on posix-aio w/o patch |184.0 | 80.46 | 6.35
> >kvm: cache=on posix-aio w/ patch |165.0 | 70.90 | 7.09
> >------------ new results
> >----------+------+------------------+------------------
> >kvm:cache=off posix-aio fd_pool[16]| 78.2 | 58.24 | 15.43
> >kvm:cache=off posix-aio fd_pool[64]|129.0 | 71.62 | 9.11
> >
>
> That's a nice result. We could probably improve the latency by tweaking
> the queue sizes.
Yeah, I was quite pleased to see a simpler solution perform so well.
>
> Very nice work! Thanks for doing the thorough analysis.
Thanks, very happy to see a signficant improvement in IO here.
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
address@hidden