Re: [libmicrohttpd] connections reply nothing when using libmicrohttpd

[Christian Grothoff]

Hi,

Might it be that you are using 'select()' in your event loop? (The rest
of the answer does not apply if you do not, but it is almost certain
that this is your problem).  There, the FD_SIZE limit applies (not just
the ulimit), so once you hit near 1000 FDs open from mmap, the server
won't be able to accept() incoming connections and pass them to select()
anymore.  As MHD accepted(), then got a "bad" (high) FD, its only choice
is to close() the FD, which leads exactly to the behavior your describe.


In this case, you have three possible solutions:
1) change to poll or epoll, also improving performance and scalabiltiy
2) Increase your ulimit (as you did presumably in point 4) and then
   use 'dup2' for the files you open for mmap() to move the FD into
   the range *above* 1024.  Make sure to close() the file FDs below 1024
   afterwards.  That way, MHD can keep using the low FDs for the TCP
   sockets.
3) Depending on your situation, you might also be able to simply
   open the file, call mmap(), then *close* the file but not call
   munmap() until the response is done.  That way, you don't burn an
   FD at all and don't need to do either (1) or (2).


Note that once you have more than 1000 concurrent network connections,
only (1) will really fix your problems.


Happy hacking!

Christian

On 10/21/2017 10:38 AM, Yatong Zhang wrote:
> Hi Christian,
> I built a http server with libmicrohttpd and it serves as a file content
> server. My http server opens lots of files using 'mmap' and the files
> increase daily. At first the server works fine but recently when the
> files opened about more than 1000, it begins to reply nothing. curl
> indicates that 'Server returned nothing (no headers, no data)' or
> 'Failure when receiving data from the peer‘.  So,
> 1. When the files opened are more then 1000, server sometimes close
> connections immediately. It's 'sometimes', not 'always'
> 2. The files are opened with mmap.
> 3. If the files are reduced to about 950, server works fine.
> 4. I have modified system limits, such as number opened files, but don't
> help.
> Here the limits of the server process:
> 
>     cat /proc/10758/limits
>     Limit                     Soft Limit           Hard Limit          
>     Units    
>     Max cpu time              unlimited            unlimited           
>     seconds  
>     Max file size             unlimited            unlimited           
>     bytes    
>     Max data size             unlimited            unlimited           
>     bytes    
>     Max stack size            8388608              unlimited           
>     bytes    
>     Max core file size        0                    0                   
>     bytes    
>     Max resident set          unlimited            unlimited           
>     bytes    
>     Max processes             514482               514482              
>     processes
>     Max open files            65535                65535               
>     files    
>     Max locked memory         65536                65536               
>     bytes    
>     Max address space         unlimited            unlimited           
>     bytes    
>     Max file locks            unlimited            unlimited           
>     locks    
>     Max pending signals       514482               514482              
>     signals  
>     Max msgqueue size         819200               819200              
>     bytes    
>     Max nice priority         0                    0                   
>     Max realtime priority     0                    0                   
>     Max realtime timeout      unlimited            unlimited            us 
> 
> 
> I have tuned all the parameters I can find including 'sysctl' stuff,
> nothing helps. I know this must have something to do with the number of
> opened files but I don't know why and how to fix it. So would you please
> help and give me some advices?
> Thanks very much.
> 
> Best regards,
> 
> Yatong Zhang

signature.asc
Description: OpenPGP digital signature