[libmicrohttpd] RE: MHD_handle_connection() performance issue

[Eivind Sarto]

I going to give some feedback to my own posting :-)

I think the solution is that MHD_handle_connection() should not set the POLLIN 
flag for the listen/server socket.
It should change the code to the following:
           fd = con->daemon->socket_fd;
           p[1].fd = fd;
//         p[1].events |= POLLIN;     /*OLD*/
           p[1].events = 0;                  /*NEW*/

Even if pollfd.events=0, pollfd.revents=POLLHUP when the daemon socket is shout 
down.   And that is what we care about, right?
We don't want to know if there is any input data on the daemon socket.

I looked at the linux kernel source and it will set pollfd.revents=POLLHUP when 
the socket is closed, regardless of the value
of  pollfd.events.  Not sure if every OS behaves this way, but maybe someone 
who knows other OS can provide some feedback.

Ultimately, it would probably be better (faster) to only poll a single fd (the 
connection socket) and find some other way to
detect if the server is shutting down,  but this change makes a huge difference 
in system load with lots of connections.

-eivind
________________________________________
From: address@hidden address@hidden On Behalf Of Eivind Sarto address@hidden
Sent: Tuesday, March 22, 2011 5:16 PM
To: address@hidden
Subject: [libmicrohttpd] MHD_handle_connection() performance issue

I am using libmicrohttpd to serve a large number of video players that is using 
a "smooth" streaming format.
That means the video stream is divided into separate chunks and that the client 
can decide to jump to a
faster or slower encoding of the video stream on any chunk boundary.  The 
length of a video chunk is
typically several seconds (<= 10 sec).
(If you have 1000 clients all playing videos in 10 second chunks, you will have 
100 requests/sec)

I wrote a client simulator so I can do some load testing without buying a whole 
bunch of client devices.

The server gets extremely busy as crank up the number of clients.  I am running 
the server with
   MHD_USE_THREAD_PER_CONMHDNECTION | MHD_USE_POLL
The reason for this appears to be that MHD_handle_connection() will poll both 
the connection socket and
the listen/server socket.  And every time there is a new connection, all the 
active connections will return
from poll, because they are also polling the listen/server socket.
It gets pretty awful as you try and crank up 1000 simulated streams.  The cpus 
are almost completely saturated with
each connection/thread polling again and again, each time a new connection is 
made.

I made a debug version of  MHD_handle_connection() that only polls the 
connection socket, and I can run several
1000 video clients with almost no cpu load.
But, each connection thread can no longer detect if the server is shutting down.

Any suggestion or feedback about what could be done to improve this situation?

Note: I am using the latest svn version with the fixes to MHD_poll() for 
MHD_USE_THREAD_PER_CONMHDNECTION.

-eivind