libmicrohttpd
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [libmicrohttpd] ContentReaderCallback has changed behaviour in lates


From: Christian Grothoff
Subject: Re: [libmicrohttpd] ContentReaderCallback has changed behaviour in latest releases
Date: Fri, 01 Feb 2013 10:40:44 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.10) Gecko/20121027 Icedove/10.0.10

Please look at src/examples/fileserver_example_external_select.c

The main point is that *you* do the 'select' call in your main loop and just ask MHD for select sets and timeout values. In your response callback, you can then safely return '0' if you don't have any data ready (which will cause MHD to remove the respective FD from the select set for the next MHD_get_fdset call). You are then responsible for adding a timeout/socket/etc. to your external select call to ensure that MHD is woken up if your response callback is (possibly) ready again.

External select is the default if you do not pass any other threading mode option, thus you don't find it as a MHD_USE option...

Happy hacking!

-Christian

On 02/01/2013 10:36 AM, David J Myers wrote:
Could you explain 'external_select'? Is this a different threading model
from 'thread-per-connection'? I can't see a parameter definition for this.
Could you show me an example call to MHD_start_daemon which selects this
mode?
Regards
David


-----Original Message-----
From: Christian Grothoff [mailto:address@hidden
Sent: 01 February 2013 09:08
To: David J Myers; libmicrohttpd development and user mailinglist
Subject: Re: ContentReaderCallback has changed behaviour in latest releases

On 02/01/2013 10:00 AM, David J Myers wrote:
Ok, I changed the threading mode from MHD_USE_SELECT_INTERNALLY (with
thread
pool) to MHD_USE_THREAD_PER_CONNECTION and this fixes the problem. I
guess I was getting away with it on previous releases.
Possible -- I checked and found that the error message has been there for a
long time (0.4.x or even earlier).  However, you might have had it disabled
(configure defaults changed at some point), and MHD might have been more
tolerant of busy waiting in the past.

I'm hoping that this mode
change will not have any unwanted effects on the behaviour of the
system (IP video camera).
I don't see why it would, except that you get more threads which may consume
more memory.  Note that if you use 'thread per connection', the idea is that
your code should just block instead of returning zero. If you're concerned
about performance (i.e. RAM) on an embedded system, 'external select' is
usually the best choice IMO. Thread-per-connection can be fine if your
application logic is complex and you'd rather be able to block while putting
the response together, but it is generally not the best choice for
performance.

Happy hacking!

-Christian

Thanks, Christian.
Best regards
David



-----Original Message-----
From: Christian Grothoff [mailto:address@hidden
Sent: 31 January 2013 17:53
To: David J Myers
Cc: address@hidden
Subject: Re: ContentReaderCallback has changed behaviour in latest
releases

On 01/31/2013 05:43 PM, David J Myers wrote:
Hi Christian,

I have a problem with the latest releases of libmicrohttpd

Previously I was using version 0.9.7 and I have recently upgraded to
0.9.20 and I have also checked this in 0.9.24

I use a ContentReader Callback to chunk my video data

response = MHD_create_response_from_callback (MHD_SIZE_UNKNOWN,

                                                  1024,

&crc, responseptr,&crcf);

Sometime, in the callback I throw away a frame of data, so I return
zero from the callback, like this.

static ssize_t
crc (void *cls, uint64_t pos, char *buf, size_t max) {

.

.

.

if (NothingToDo)

{

      return 0; // MHD should call me back later

}

else

{

    //do stuff

.

.

.

      return no_of_bytes_processed;

}

}

If I return zero from the callback I expect MHD to call me back some
short time later.

This has always worked for me in the past but now, when I return
zero, from the callback, version 0.9.20 does not call me back until
after a delay of around 6 seconds by which time the client has timed out.
Version 0.9.24 is worse and actually crashes the program with the
following error:-

Fatal error in GNU libmicrohttpd connection.c:395: API violation


Aborted

Is it no longer allowed to return zero from a ContentReaderCallback?


That depends on your threading mode.  From the documentation:

/**
    * Callback used by libmicrohttpd in order to obtain content.  The
    * callback is to copy at most "max" bytes of content into "buf".  The
    * total number of bytes that has been placed into "buf" should be
    * returned.<p>
    *
    * Note that returning zero will cause libmicrohttpd to try again,
    * either "immediately" if in multi-threaded mode (in which case the
    * callback may want to do blocking operations) or in the next round
    * if MHD_run is used.  Returning 0 for a daemon that runs in internal
    * select mode is an error (since it would result in busy waiting) and
    * will cause the program to be aborted (abort()).
    *
    * @param cls extra argument to the callback
    * @param pos position in the datastream to access;
    *        note that if an MHD_Response object is re-used,
    *        it is possible for the same content reader to
    *        be queried multiple times for the same data;
    *        however, if an MHD_Response is not re-used,
    *        libmicrohttpd guarantees that "pos" will be
    *        the sum of all non-negative return values
    *        obtained from the content reader so far.
    * @param buf where to copy the data
    * @param max maximum number of bytes to copy to buf (size of buf)
    * @return number of bytes written to 'buf';
    *  0 is legal unless we are running in internal select mode (since
    *    this would cause busy-waiting); 0 in external select mode
    *    will cause this function to be called again once the external
    *    select calls MHD again;
    *  MHD_CONTENT_READER_END_OF_STREAM (-1) for the regular
    *    end of transmission (with chunked encoding, MHD will then
    *    terminate the chunk and send any HTTP footers that might be
    *    present; without chunked encoding and given an unknown
    *    response size, MHD will simply close the connection; note
    *    that while returning END_OF_STREAM is not technically
    *    legal if a response size was specified, MHD accepts this
    *    and treats it just as MHD_CONTENT_READER_END_WITH_ERROR;
    *  MHD_CONTENT_READER_END_WITH_ERROR (-2) to indicate a server
    *    error generating the response; this will cause MHD to simply
    *    close the connection immediately.  If a response size was
    *    given or if chunked encoding is in use, this will indicate
    *    an error to the client.  Note, however, that if the client
    *    does not know a response size and chunked encoding is not in
    *    use, then clients will not be able to tell the difference between
    *    MHD_CONTENT_READER_END_WITH_ERROR and
MHD_CONTENT_READER_END_OF_STREAM.
    *    This is not a limitation of MHD but rather of the HTTP protocol.
    */


So returning 0 is illegal if you're using MHD's internal select mode,
as we don't want applications to do busy-waiting.  It is quite
possible that we changed the library behavior here a bit, to perform more
stringent checking.
There was a change related to that logic between
0.9.13 and 0.9.14.

If you can tell me what threading mode you're using (or even have a
small example), that would be helpful to find out more about what's going
on.
Happy hacking!

Christian
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.2890 / Virus Database: 2639/6069 - Release Date:
01/30/13

-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.2897 / Virus Database: 2639/6072 - Release Date: 01/31/13





reply via email to

[Prev in Thread] Current Thread [Next in Thread]