Re: [libmicrohttpd] Bench-marking issue

[silvioprog]

Hello Christ, thanks a lot for the clarifications.

You are absolutely right! After applying some basic optimizations, I got an impressing result with MHD. Please check this new log below (faster first):

MHD:

Running 30s test @ http://127.0.0.1:8080/

  12 threads and 400 connections

  Thread Stats   Avg      Stdev     Max   +/- Stdev

    Latency     4.06ms    5.82ms 198.70ms   87.22%

    Req/Sec    14.36k     4.85k   60.51k    85.61%

  5147798 requests in 30.09s, 490.93MB read

Requests/sec: 171061.39

Transfer/sec:     16.31MB

Mongoose:

Running 30s test @ http://127.0.0.1:8080/

  12 threads and 400 connections

  Thread Stats   Avg      Stdev     Max   +/- Stdev

    Latency     3.23ms    2.11ms 203.59ms   99.65%

    Req/Sec    10.41k     1.18k   32.95k    96.58%

  3720295 requests in 30.09s, 358.34MB read

Requests/sec: 123638.94

Transfer/sec:     11.91MB

Soup:

Running 30s test @ http://127.0.0.1:8080/

  12 threads and 400 connections

  Thread Stats   Avg      Stdev     Max   +/- Stdev

    Latency    41.11ms  134.98ms   1.90s    91.37%

    Req/Sec     4.15k     6.60k   18.86k    80.22%

  510925 requests in 30.09s, 67.24MB read

  Socket errors: connect 0, read 0, write 0, timeout 46

Requests/sec:  16981.30

Transfer/sec:      2.23MB

New tests in attachment, now you just need:

$ make

$ make test

That's all! :-)

On Sat, Jan 28, 2017 at 5:23 PM, Christian Grothoff <address@hidden> wrote:

Hi!

I don't know why you used the rather castrated setup for MHD for
benchmarking.

First of all, for benchmarking you should only create the response
object once (in main), and only destroy it once (in main), and re-use it
across connections in your 'ahc_echo'.  That might give you some cheap
boost.

Second, you are using 'SELECT', but depending on the platform
EPOLL might be better. Similarly, you're only giving one core/thread to
MHD (no thread pool), not sure what Mongoose/Soup are using without any
options.  So the comparisson may or may not be fair here.

Finally, I can't see if you compiled MHD (or the other servers) with any
special CFLAGS, i.e. O2/O3/Os or even O0/O1.  So that might also make
some small difference.

Happy benchmarking!

Christian

On 01/28/2017 09:05 PM, silvioprog wrote:
> Hello masters,
>
> I'm doing some bench-markings comparing performance of popular libraries
> like MHD (my favorite ❤), libsoup (failed in basic tests), mongoose
> (fastest), nginx (echo module), kore, etc. etc. etc., however, I'm not sure
> about the configuration I used in MHD application. Please check the results
> below (mongoose/mhd/soup):
>
> Mongoose:
> Running 30s test @ http://127.0.0.1:8080/
>   12 threads and 400 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency     3.20ms    3.01ms 206.15ms   99.82%
>     Req/Sec    10.58k     1.24k   24.71k    96.61%
>   3793415 requests in 30.08s, 365.39MB read
> Requests/sec: 126094.30
> Transfer/sec:     12.15MB
>
> MHD:
> Running 30s test @ http://127.0.0.1:8080/
>   12 threads and 400 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency     8.71ms    6.11ms 318.04ms   99.59%
>     Req/Sec     3.90k   360.50     8.74k    96.45%
>   1390656 requests in 30.07s, 132.63MB read
> Requests/sec:  46250.87
> Transfer/sec:      4.41MB
>
> Soup:
> Running 30s test @ http://127.0.0.1:8080/
>   12 threads and 400 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency    55.37ms  206.43ms   1.97s    92.87%
>     Req/Sec     4.85k     6.93k   18.41k    76.75%
>   510046 requests in 30.03s, 67.13MB read
>   Socket errors: connect 0, read 0, write 0, timeout 97
> Requests/sec:  16982.12
> Transfer/sec:      2.23MB
>
> Attached all files used to test them (mongoose.c can be downloaded from
> https://github.com/cesanta/mongoose). My environment:
>
> Xubuntu 16.04 64 bits (6 GB RAM / SSD Kingston)
> Linux silvioprog-dev 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC
> 2017 x86_64 x86_64 x86_64 GNU/Linux
> $ pkg-config libsoup-2.4 glib-2.0 libmicrohttpd --modversion
> 2.52.2
> 2.48.1
> 0.9.52
> configure: libmicrohttpd 0.9.52 Configuration Summary:
>   Cross-compiling:   no
>   Operating System:  linux-gnu
>   Threading lib:     posix
>   Use thread names:  yes
>   Inter-thread comm: eventfd
>   libcurl (testing): yes
>   Target directory:  /usr/local
>   Messages:          yes
>   Basic auth.:       yes
>   Digest auth.:      yes
>   Postproc:          yes
>   HTTPS support:     yes (using libgnutls and libgcrypt)
>   poll support:      yes
>   epoll support:     yes
>   build docs:        yes
>   build examples:    yes
>
> So, I have a question: is my configuration ideal for benchmark testing?
> (I'm not sure about soup/mongoose configuration too, but it seems OK).
>
> Build and test instruction are in the attached file too.
>
> Thank you!
>
> --
> Silvio Clécio

--

Silvio Clécio

bench-II.tar.gz
Description: GNU Zip compressed data