[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TI
From: |
Christian Grothoff |
Subject: |
Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode |
Date: |
Mon, 7 Dec 2015 21:27:29 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Icedove/38.4.0 |
On 12/07/2015 08:58 PM, silvioprog wrote:
> Hello, I have good news in the test 2, however in test 3 ...
>
> @Christ, on Linux I got 0.00% erros. MHD is so fast on Linux that I believe
> that it can support C10K
> <https://www.nginx.com/resources/wiki/?highlight=c10k> problem. :-)
Well, on Linux C10k is easy these days. We had people using MHD that in
practice had what you might call the C100k problem, and that was doable ;-).
> @Evg, do I need to change the `FD_SETSIZE` and the MHD source before
> compiling or in the `CMakeLists.txt` of my example?
>
> So, I did more tests after read Evg's messages. Now all results that I'll
> show uses keep-alive. Test 1:
>
> MHD_start_daemon(
> MHD_USE_THREAD_PER_CONNECTION | MHD_USE_DEBUG, PORT, NULL, NULL,
> &answer_to_connection, NULL,
> MHD_OPTION_END);
>
>
> 40~50% erros. And it returns a lot of `Server reached connection limit
> (closing inbound connection)` and some `Server reached coServer reached
> connection limit (closing inbound connection)`, OK, after read this log, I
> did:
>
> Test 2 (thanks Evgeny):
>
> MHD_start_daemon(MHD_USE_THREAD_PER_CONNECTION | MHD_USE_DEBUG, PORT, NULL,
> NULL,
> &answer_to_connection, NULL,
> * MHD_OPTION_CONNECTION_TIMEOUT, (unsigned int) 240,*
> * MHD_OPTION_CONNECTION_LIMIT, (unsigned int) 16000,*
> MHD_OPTION_END);
>
>
> *Samples: 15000.*
> *0,00% erros.*
> *95% line: 3.*
> *99% line: 5.*
>
> OMG, now MHD is so fast! :-o
>
> But, make this small change in your JMeter config:
>
> https://dl.dropboxusercontent.com/u/135304375/snip1.png (I can send a new
> jmx file if you prefer)
>
> and in the MHD example:
>
> * MHD_OPTION_CONNECTION_LIMIT, (unsigned int) 16000 * 2,*
>
> 1,84% erros. And a lot of: `Failed to create a thread: Not enough space`.
> OK, so I disabled the "thread per connection":
Well, thread-per-connection was never the most scalable option,
especially if used like this...
> MHD_start_daemon(*MHD_USE_SELECT_INTERNALLY *| MHD_USE_DEBUG, PORT, NULL,
> NULL,
> &answer_to_connection, NULL,
> MHD_OPTION_CONNECTION_TIMEOUT, (unsigned int) 240,
> * MHD_OPTION_CONNECTION_LIMIT, (unsigned int) 16000 * 2,*
> MHD_OPTION_END);
>
>
> *Samples: 27255.*
> *Error: 99,61%. LOL*
>
> And it locks the last requests.
If you really need to hit MHD with thousands of parallel requests, you
may want to consider using the thread *pool* (set the thread pool size).
> Sorry for my ignorance, but MHD_USE_SELECT_INTERNALLY should be the more
> fast MHD option, then or I did something wrong in my test, or this MHD
> option has a small bug on Windows. :-/
Not necessarily. Using thread-per-connection will use way more RAM, but
fewer expensive system calls. So until you hit your memory limit or run
into scheduler problems, thread-per-connection can be more scalable,
especially on W32 where the current 'select()'-mechanism is rather
awkward AFAIK.
But even on GNU/Linux thread-per-connection can be faster as long as
your system can handle all the threads (and doesn't encounter trouble
with cache misses for all the stacks).
> ps. I need to test the `FD_SETSIZE` option, so I'll wait for Evg's answer.
> :-)
>
> On Mon, Dec 7, 2015 at 4:53 AM, Christian Grothoff <address@hidden>
> wrote:
>
>> Hi!
>>
>> I tried your script on my system, with keep-alive, and I get 0.01%
>> errors (throughput: 5395.7/sec, 860.2 kb/s for 15000 samples).
>>
>> If I change the code to add the "close" header, I get 0.13% errors,
>> 3425/sec throughput, and 535kb/s, which I'd say is expected.
>>
>> So this seems to be (again) a W32-specific issue :-(.
>>
>> -Christian
>
>
signature.asc
Description: OpenPGP digital signature
- [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/04
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/04
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, Christian Grothoff, 2015/12/05
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, Evgeny Grin, 2015/12/05
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/06
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, Evgeny Grin, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, Evgeny Grin, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, Evgeny Grin, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, silvioprog, 2015/12/08
- Re: [libmicrohttpd] Problem with Keep-Alive and MHD_OPTION_CONNECTION_TIMEOUT in MHD_USE_THREAD_PER_CONNECTION mode, Evgeny Grin, 2015/12/08