libmicrohttpd
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [libmicrohttpd] ContentReaderCallback has changed behaviour in lates


From: Christian Grothoff
Subject: Re: [libmicrohttpd] ContentReaderCallback has changed behaviour in latest releases
Date: Mon, 11 Feb 2013 13:44:28 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.10) Gecko/20121027 Icedove/10.0.10

There are not really 'thread handles' on GNU systems (but -- are you on GNU or on W32? Either system has some limits, but they are typically so high that you should never hit them).

If this is a GNU/Linux system, the most likely cause is that you're out of memory. You may try to set the size of the stack per thread to a smaller value (at your own peril if you get a stack overflow...). This can be done using MHD_OPTION_THREAD_STACK_SIZE.

Threads should be destroyed by MHD after use, but memory _fragmentation_ can still cause the OS to be unable to find a suitably large contiguous block of memory for a thread --- depending in particular on your application's memory allocation behavior. Analyzing memory fragmentation issues is extremely tricky in my experience, but a major research funding agency told me a few years ago that writing tools to help developers do so wasn't research, so I cannot point you to any good method for diagnosing such problems.

Note that reducing the per-thread stack size is unlikely to fix memory fragmentation issues -- it might just take longer for them to manifest. Setting a smaller per-thread stack size is mostly useful on systems with limited memory, well-understood stack requirements and a large number of concurrent connections in relation to system memory.

Happy hacking!

-Christian

On 02/11/2013 12:56 PM, David J Myers wrote:
Since switching to MHD_USE_THREAD_PER_CONNECTION, after the system has been
running for a number of hours, I'm getting the following error message:-
"Failed to create a thread: Resource temporarily unavailable"
I can't tell if this a shortage of memory resources or thread handles or
what.
Is there something I need to do to cause threads to be destroyed after use?
BR
David



-----Original Message-----
From: Christian Grothoff [mailto:address@hidden
Sent: 01 February 2013 10:55
To: David J Myers
Cc: 'libmicrohttpd development and user mailinglist'
Subject: Re: ContentReaderCallback has changed behaviour in latest releases

On 02/01/2013 11:32 AM, David J Myers wrote:
My server must stream to a number of concurrent clients. Does that not
therefore mean that I need a multi-threaded mode, not the
single-threaded external-select mode?
External-select can handle up to 1k concurrent clients.
If I use the thread-per_connection mode, what is the consequence of
returning zero from the callback instead of blocking? Does it create a
new thread for the next callback?
No, MHD will just call the callback instantly again.
If I use the thread-pool mode, I can't return zero from the callback,
but what would be the consequence of blocking in this mode? This is my
preferred mode.
In thread-pool mode, blocking would cause all other connections that are
processed by the same thread to stall.  So that's the worst kind of behavior
as the other connections might have data ready.  Thus, the thread pool mode
must only be used if the callbacks are "virtually non-blocking" (by
"virtually", I mean to say that I'd not necessarily count disk IO operations
in this).  Note that *all* modes can support concurrent clients; the only
fundamental limitation (with respect to
performance) of the select/poll modes is that they only use a single core.


Happy hacking!

Christian
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.2897 / Virus Database: 2639/6072 - Release Date: 01/31/13





reply via email to

[Prev in Thread] Current Thread [Next in Thread]