Re: [libmicrohttpd] Re : Re: External select mode limited to 1024 FDs

[Christian Grothoff]

On 03/11/2015 12:58 PM, address@hidden wrote:
> Hi Christian,
> 
> I wanted to use external threads because my system has the following 
> constraints:
> 1) One connection per thread: the response to the client is time critical and 
> I cannot afford that a connection delays another one.

Then you should use the one-thread-per-connection mode, that is exactly
what you want.  MHD will use a thread for each connection, which matches
your requirements.  This method also has no limits on open FDs as MHD
doesn't use select() but blocking read/write operations instead.

> 2) Many listening sockets are opened, hence many MHD daemon: some for HTTP 
> and some for HTTPS.
> 3) Up to 2000 concurrent connections (shared between HTTP and HTTPS).

Well, 2000 isn't much, so one-thread-per-connection should be fine.

> As I don't know how the traffic will be shared amongst the daemons, I would 
> need to configured each MHD daemon with epoll and 2000 threads.
> This solution does not scale. Just imagine the number of threads created at 
> start up or debugging a single HTTP request with gdb.

2000 threads is fine on modern systems, and if you really run into
problems, you can easily switch to thread-pool-with-epoll.  As for
debugging, well, that's always an issue if you have 2000 concurrent
actions ;-).

> The best solution would be to share the worker threads pool between the 
> daemons but if I understand correctly the current implementation does not 
> allow it.

Well, if you use on thread per connection, there is no sharing anyway.
If you have HTTP and HTTPS, you are right right now you'd have two
daemons and no joint pool. Two solutions: you can setup a reverse proxy
using Apache or nginx to forward HTTPS to HTTP, or just live with a
slight imbalance. I suspect the reverse proxy is the best option, as
then you can even move that to another physical machine easily, and
you're off-loading HTTPS.

> This why I fall back on external thread: I create 2000 daemons for each 
> listening socket and dispatch the HTTP/HTTPS request to a pool of 2000 
> threads, which reuse the daemons.
> 
> Is there a plan to implement worker threads sharing between daemons? If not, 
> are there other MHD solutions that I miss?

I don't have a plan to do that, seems a bit overengineered.  For this
stuff, I'd rather keep it simple unless given hard proof that this is a
major issue.  I don't think without having performance measurements one
should even worry about this.


Happy hacking!

Christian

> Thanks,
> 
> Louis
> 
> Le 05/03/15, Christian Grothoff  <address@hidden> a écrit :
>> Hi!
>>
>> I had no plans for an 'external poll' API, as we have epoll() which is
>> better for very large FDsets (as long as you're on Linux). Also,
>> redefining FD_SETSIZE is also IMO a very good possibility.
>>
>> Regardless, select/poll with > 1024 FDs and external mode is likely not
>> the greatest idea from a scalability/performance perspective: if you
>> have that many concurrent / open FDs, you really want epoll() and
>> possibly think about using threads / thread pool for processing.
>>
>> Happy hacking!
>>
>> Christian
>>
>> On 03/04/2015 10:22 PM, address@hidden wrote:
>>> Hi,
>>>
>>> I would like to use the external select mode, as I need to manage the 
>>> connection threads externally.
>>> Because there could be much more than 1024 FDs used at run time, I cannot 
>>> use MHD_get_fdset() and select ().
>>> Apart from redefining FD_SETSIZE, are there other ways to use external 
>>> threads when FD > 1024?
>>> If not, is there a plan to add an MHD_get_poll_event API, the poll version 
>>> of MHD_get_fdset()?
>>>
>>> Thanks,
>>>
>>> Louis Benoit
>>>
>>
>>
>>
>