[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[libmicrohttpd] RE: MHD_handle_connection() performance issue

From: Eivind Sarto
Subject: [libmicrohttpd] RE: MHD_handle_connection() performance issue
Date: Tue, 22 Mar 2011 21:06:23 -0400

I going to give some feedback to my own posting :-)

I think the solution is that MHD_handle_connection() should not set the POLLIN 
flag for the listen/server socket.
It should change the code to the following:
           fd = con->daemon->socket_fd;
           p[1].fd = fd;
//         p[1].events |= POLLIN;     /*OLD*/
           p[1].events = 0;                  /*NEW*/

Even if, pollfd.revents=POLLHUP when the daemon socket is shout 
down.   And that is what we care about, right?
We don't want to know if there is any input data on the daemon socket.

I looked at the linux kernel source and it will set pollfd.revents=POLLHUP when 
the socket is closed, regardless of the value
of  Not sure if every OS behaves this way, but maybe someone 
who knows other OS can provide some feedback.

Ultimately, it would probably be better (faster) to only poll a single fd (the 
connection socket) and find some other way to
detect if the server is shutting down,  but this change makes a huge difference 
in system load with lots of connections.

From: address@hidden address@hidden On Behalf Of Eivind Sarto address@hidden
Sent: Tuesday, March 22, 2011 5:16 PM
To: address@hidden
Subject: [libmicrohttpd] MHD_handle_connection() performance issue

I am using libmicrohttpd to serve a large number of video players that is using 
a "smooth" streaming format.
That means the video stream is divided into separate chunks and that the client 
can decide to jump to a
faster or slower encoding of the video stream on any chunk boundary.  The 
length of a video chunk is
typically several seconds (<= 10 sec).
(If you have 1000 clients all playing videos in 10 second chunks, you will have 
100 requests/sec)

I wrote a client simulator so I can do some load testing without buying a whole 
bunch of client devices.

The server gets extremely busy as crank up the number of clients.  I am running 
the server with
The reason for this appears to be that MHD_handle_connection() will poll both 
the connection socket and
the listen/server socket.  And every time there is a new connection, all the 
active connections will return
from poll, because they are also polling the listen/server socket.
It gets pretty awful as you try and crank up 1000 simulated streams.  The cpus 
are almost completely saturated with
each connection/thread polling again and again, each time a new connection is 

I made a debug version of  MHD_handle_connection() that only polls the 
connection socket, and I can run several
1000 video clients with almost no cpu load.
But, each connection thread can no longer detect if the server is shutting down.

Any suggestion or feedback about what could be done to improve this situation?

Note: I am using the latest svn version with the fixes to MHD_poll() for 


reply via email to

[Prev in Thread] Current Thread [Next in Thread]