Re: [libmicrohttpd] questions on performance

[Christian Grothoff]

Hi!

First of all, it would be much easier to help if you could make your 
benchmark available; that way, it becomes easier to see what might be
wrong.

Second, the (1) setting should be fine for performance, even though
40 threads might be a bit much unless you really have 40 cores.

Finally, I'm not sure if what you're measuring is actually what matters 
--- while MHD should call the 'completion' callback as soon as it is 
done, the client might see requests being completed much faster. 
Usually, when I do benchmarks, I looked at how fast the client got 
answers.

Now, I'm not saying looking at the completion callback is a bad idea --- 
 find it quite interesting that there is this significant delay.  A few 
years ago we had a bug filed against MHD for calling the
completion callback 'too late' (i.e. with 1 s delay, IIRC); maybe you 
have hit a corner case where this issue still exists.  To understand 
this better, it would be useful to know how fast the clients receive the 
replies --- and to see the source for your benchmark...


Happy hacking!

-Christian


On 05/28/13 05:51, Taehwan Weon wrote:
> Hi,
>
> I made a caching proxy based on libmicrohttpd 0.9.27
> While doing a stress test with tsung, I got a strange problem.
>
> My daemon start params were
> 1) MHD_USE_SELECT_INTERNALLY with # of thread = 40
> 2) MHD_USE_POLL|MHD_USE_THREAD_PER_CONNECTION
>
> My stress tool(tsung) is generating 10,000 GET requests/sec for 1,000
> 1KB objects with protocol of HTTP 1.1
> HTTP 1.1 Keep-alive feature was off ('cause tsung does not support)
>
> My measurement is between the time when the first request bytes and
> the time when completion notification callback signaled.
> I subtracted the internal processing time (file-IOs)
>
> Please look at the following values (measured for 2-hours)
> 1. elapsed time between the time request handler called and time time
> completion notification called back. : about 900~1100 msec/req
> 2. internal file-io(cache-based) operation time : 3 msec/req (all
> objects were cached in main memory)
>      (to measure file-IO time, I measured the processing time of the
> callback given to MHD_create_response_from_callback)
>
> In the 2nd param(MHD_USE_POLL), I found about more than 1,000
> libmicrohttpd threads.
>
> Could you please recommend me what I can do for better performance?
>
> -----
> weon