Re: [Qemu-devel] [PATCH v2 0/6] aio: Support epoll by introducing qemu_poll abstraction

[Fam Zheng]

On Thu, 12/04 11:43, Fam Zheng wrote:
> v2: Emulate nanoseconds precison of timeout with ppoll and timerfd.
>     Their performance is on par with each other, but both much better than
>     qemu.git:
> 
>     syscall         high # of fd      low # of fd
>     -------------------------------------------------
>     qemu.git(ppoll) 44                96
>     ppoll+epoll     85                101
>     timerfd+epoll   87                109

More data points.

Xiaomei tested this series (applied on top of RHEL 7 qemu-kvm-rhev) and found
that:

0) when # of fds is high, epoll solutions are much better (+30%).

1) timerfd+epoll is slightly better than ppoll+epoll, but the difference is
minimal.

2) original code is 2%~5% faster than the new implementations when # of fds is
low.

This leads to the conclusion that that we'll have a small performance
degradation if merge this series. I'm thinking about possible optimizations.
Options in my mind are:

1) Remove 1ns PR_SET_TIMERSLACK in timerfd+epoll, this doesn't make qemu_poll
faster than the old qemu_poll_ns, but may have other positive effects that
compensate the cost.

2) Use dynamic switch between ppoll and timerfd+epoll. In poll-linux.c, We
start with pure ppoll, while keeping track of elapsed time in ppoll. And
periodically, we try "timerfd+epoll" for a few iterations, so that we can
compare if it is faster than pure ppoll. If it is, swap them, use timerfd+epoll
and and periodically try "ppoll".

That said, I'll also look at the kernel side. Maybe optimizing ppoll or just
add EPOLL_NANOSECOND_TIMEOUT to epoll_create1 is a better place for
engineering.

Fam

> 
> (In high # of fd case, 3 activated but idle virtio-console devices are
> attached, which will add us hundereds of fds to poll)
> 
> v1 cover letter
> ---------------
> 
> ppoll(2) doesn't scale as well as epoll: The elapsed time of the syscall is
> linear to the number of fd's we poll, which hurts performance a bit when the
> number of devices are many, or when a virtio device registers many virtqueues
> (virtio-serial, for instance).
> 
> This series introduces qemu_poll, which is implemented  with g_poll and epoll,
> decided at configure time with CONFIG_EPOLL.
> 
> Fam
> 
> 
> Fam Zheng (6):
>   poll: Introduce QEMU Poll API
>   posix-aio: Use QEMU poll interface
>   poll: Add epoll implementation for qemu_poll
>   main-loop: Replace qemu_poll_ns with qemu_poll
>   tests: Add test case for qemu_poll
>   poll-linux: Add timerfd support
> 
>  Makefile.objs           |   2 +
>  aio-posix.c             |  52 ++++----
>  async.c                 |   5 +-
>  include/block/aio.h     |   7 +-
>  include/qemu/poll.h     |  40 ++++++
>  include/qemu/timer.h    |  13 --
>  include/qemu/typedefs.h |   2 +
>  main-loop.c             |  35 +++++-
>  poll-glib.c             | 130 ++++++++++++++++++++
>  poll-linux.c            | 314 
> ++++++++++++++++++++++++++++++++++++++++++++++++
>  qemu-timer.c            |  28 -----
>  tests/Makefile          |   2 +
>  tests/test-poll.c       | 272 +++++++++++++++++++++++++++++++++++++++++
>  13 files changed, 821 insertions(+), 81 deletions(-)
>  create mode 100644 include/qemu/poll.h
>  create mode 100644 poll-glib.c
>  create mode 100644 poll-linux.c
>  create mode 100644 tests/test-poll.c
> 
> -- 
> 1.9.3
> 
>