Re: [libmicrohttpd] Thread Pooling

[Richard Alimi]

Hi Christian,

On Wednesday 04 March 2009 11:32:20 pm Christian Grothoff wrote:
> On Wednesday 04 March 2009 05:46:44 pm Richard Alimi wrote:
> > Hi All,
> >
> > First of all, thank you for a wonderful library! We're currently using
> > this in our implementation of P4P, which is a way for ISPs to provide
> > network information to help applications optimize network usage (e.g.,
> > peer selection in BitTorrent).
>
> Thanks.  Is this free software that we could link to from the MHD webpage?

We haven't yet made it publicly available, but I can certainly let you know 
when the time comes.

> > 1) Is there interest in merging a patch for the current thread pooling
> > implementation?  If so, I'll be more than happy to clean it up, update to
> > the current SVN revision, and submit.
> >
> > 2) Is there interest in extending it to support multiple connections
> > within each thread? If so, I can see about additionally implementing this
> > behavior.
>
> I'd say that would be interesting, especially if either
> 1) the needed growth in the code base is really small or
> 2) support to disable the feature using configure is present

Looking at the current patch (against the rev 7728), it is 286 additional 
lines of code and only a few replaced, so accomplishing conditional 
compilation should be quite simple.  It basically works by adding another 
threading model, a backlog queue, and re-using the existing code for handling 
an active connection (MHD_handle_connection).

I'll let you decide whether you think 286 lines of code qualifies as small or 
not :)

> Note that it is not quite enough to just have a patch against src/daemon/,
> I'd like to also see reasonable test cases for the new feature in testcurl/
> so that we can ascertain that it works on the various platforms.

Agreed. I'll try to add some good test cases as well.

> Overall, I'd certainly say thread-pool + select/epoll-per-thread (as well
> as a pure epoll/kqueue model) would all be great to have; however, doing
> the work load balancing between the different threads might require some
> tricky code. So in my book the quality (== simplicity) of the code and its
> test cases is really the only question that I'd raise before applying such
> a patch.

It certainly is trickier if load balancing is a target.  I'll first cleanup 
the existing thread pool patch, then consider the second (integration with 
select/epoll) as a second item.

It seems to me there could be some possible steps towards load balancing:

(1) When accepting a new connection, add it to the thread with the fewest 
number of active connections. Connections never move between threads.

(2) Allow connections to move between threads. One simple metric is to try to 
equalize the number of connections per thread.

(3) Allow the library user to influence load balancing to take into account 
the application's processing overhead per request.  I don't currently have a 
good idea for how something like this could be done in a clean way, though.

What do you think?

-- 
Richard Alimi
Department of Computer Science
Yale University