|
From: | Yatong Zhang |
Subject: | Re: [libmicrohttpd] connections reply nothing when using libmicrohttpd |
Date: | Sat, 21 Oct 2017 22:00:10 +0800 |
Hi,
Might it be that you are using 'select()' in your event loop? (The rest
of the answer does not apply if you do not, but it is almost certain
that this is your problem). There, the FD_SIZE limit applies (not just
the ulimit), so once you hit near 1000 FDs open from mmap, the server
won't be able to accept() incoming connections and pass them to select()
anymore. As MHD accepted(), then got a "bad" (high) FD, its only choice
is to close() the FD, which leads exactly to the behavior your describe.
In this case, you have three possible solutions:
1) change to poll or epoll, also improving performance and scalabiltiy
2) Increase your ulimit (as you did presumably in point 4) and then
use 'dup2' for the files you open for mmap() to move the FD into
the range *above* 1024. Make sure to close() the file FDs below 1024
afterwards. That way, MHD can keep using the low FDs for the TCP
sockets.
3) Depending on your situation, you might also be able to simply
open the file, call mmap(), then *close* the file but not call
munmap() until the response is done. That way, you don't burn an
FD at all and don't need to do either (1) or (2).
Note that once you have more than 1000 concurrent network connections,
only (1) will really fix your problems.
Happy hacking!
Christian
On 10/21/2017 10:38 AM, Yatong Zhang wrote:
> Hi Christian,
> I built a http server with libmicrohttpd and it serves as a file content
> server. My http server opens lots of files using 'mmap' and the files
> increase daily. At first the server works fine but recently when the
> files opened about more than 1000, it begins to reply nothing. curl
> indicates that 'Server returned nothing (no headers, no data)' or
> 'Failure when receiving data from the peer‘. So,
> 1. When the files opened are more then 1000, server sometimes close
> connections immediately. It's 'sometimes', not 'always'
> 2. The files are opened with mmap.
> 3. If the files are reduced to about 950, server works fine.
> 4. I have modified system limits, such as number opened files, but don't
> help.
> Here the limits of the server process:
>
> cat /proc/10758/limits
> Limit Soft Limit Hard Limit
> Units
> Max cpu time unlimited unlimited
> seconds
> Max file size unlimited unlimited
> bytes
> Max data size unlimited unlimited
> bytes
> Max stack size 8388608 unlimited
> bytes
> Max core file size 0 0
> bytes
> Max resident set unlimited unlimited
> bytes
> Max processes 514482 514482
> processes
> Max open files 65535 65535
> files
> Max locked memory 65536 65536
> bytes
> Max address space unlimited unlimited
> bytes
> Max file locks unlimited unlimited
> locks
> Max pending signals 514482 514482
> signals
> Max msgqueue size 819200 819200
> bytes
> Max nice priority 0 0
> Max realtime priority 0 0
> Max realtime timeout unlimited unlimited us
>
>
> I have tuned all the parameters I can find including 'sysctl' stuff,
> nothing helps. I know this must have something to do with the number of
> opened files but I don't know why and how to fix it. So would you please
> help and give me some advices?
> Thanks very much.
>
> Best regards,
>
> Yatong Zhang
[Prev in Thread] | Current Thread | [Next in Thread] |