Re: [libmicrohttpd] libmicrohttpd blocking response from callback

[Christian Grothoff]

Hi!

Well, 100 Mb/sec on a single core sounds about right.  I have seen about
4 GBit/sec uploads over multiple connections (cores!), which
corresponds to your 500 Mb/sec for multiple connections.  I never
considered that slow before ;-).

urlencoding is not really meant for large amounts of data (why would
anyone even DO that!?), so the significantly lower performance is
hardly surprising.   You might be able to write a faster, custom
implementation of strtoul for the 2-byte hexadecimal numbers that
are being decoded and speed it up by 2x, but decoding this will
never be fast.

Speeding up the process_value_to_boundary memcmp will be virtually
impossible.  Here, what might be possible is to use "PUT" (from
JavaScript) or otherwise convince the browser to use a more
server-friendly upload encoding --- mostly by not using an HTML form to
generate the upload.  If you use such a custom encoding (ideally:
no encoding, just raw binary data!), you would then not even
initialize the MHD_PostProcessor and instead directly process the
received "upload_data" yourself.

Happy hacking!

Christian

On 10/27/13 02:27, Adam Homolya wrote:
> Hello,
> 
> I did some profiling with callgrind today after raising the buffer size to 
> 1024 Kbyte with my application. It turned out if post data is urlencoded, 
> libmicrohttpd can do not more than ~10 Mb/sec on a new gen i7 machine, and 
> the limit is the cpu. The application spends 76% of its time within 
> MHD_http_unescape, executing strtoul (67%).
> In case of multipart data, the performance is better, can perform up to 100 
> Mb/sec while uploading with one client (~500 Mb/s for multiple connections). 
> The limit is the cpu here too. The profiling showed the application spends 
> 97% of its time in process_value_to_boundary, doing memcmp. 
> 
> Kind regards,
> Adam
> 
> -----Original Message-----
> From: Christian Grothoff [mailto:address@hidden 
> Sent: 2013. október 25. 19:54
> To: Adam Homolya
> Cc: Peter Saghelyi; Attila Body; Gabor Mihaly
> Subject: Re: libmicrohttpd blocking response from callback
> 
> On 10/25/2013 07:47 PM, Adam Homolya wrote:
>> Dear Christian,
>>
>> Thank you for your fast response, the solution is excellent.
>> I have one more problem though.
>> I've been benchmarking libmicrohttpd recently, got really good results for 
>> reading large files, using response form callback. However posting large 
>> files seem to give rather bad results, and I would like to ask if I miss 
>> something. I used post_example.c for the file upload method, using 64K post 
>> buffer size. When uploading a large file with a single client on localhost 
>> the system monitor shows one of the CPU cores are always on 100% usage (if I 
>> raise uploading processes to 8, all 8 cores are on 100%), and the upload 
>> speed is always around only 9-10Mbyte / sec which seems a bit poor for me 
>> (download speed can reach 4.5 Gbyte/sec when redirecting download result to 
>> /dev/null immediately). I don't process the post data in my application's 
>> logic, so that can't be the bottleneck. Can I boost somehow this 
>> performance? Thank you very much for your help.
> 
> I don't know how your application is encoding the POST or how your 
> application logic is handling it.
> 100% CPU at 9-10 MByte is certainly, eh, unusual for a modern machine.
> I've seen upload speeds at
> 50% of download speeds (that's normal because for uploads we don't support 
> recvfile and we have to possibly do parsing), but your case seems to be much 
> worse.  I'd check
> 1) if changing the encoding by the client for the POST data helps
> 2) if changing the buffer size (64k is not much) helps --- if you have the 
> RAM try 512 KB and
>    see if it does anything
> 3) run a profiler to see where the CPU is going to.
> 
> Happy hacking!
> 
> Christian
> 
> 
>> Kind regards
>> Adam
>>
>>
>> -----Original Message-----
>> From: Christian Grothoff [mailto:address@hidden
>> Sent: 2013. október 24. 17:39
>> To: Adam Homolya
>> Cc: Attila Body; Peter Saghelyi
>> Subject: Re: libmicrohttpd blocking response from callback
>>
>> Dear Adam,
>>
>> This was discussed on the mailinglist today and a solution was committed to 
>> SVN HEAD today as well.  Note that the documentation on the website is thus 
>> outdated with respect to this change; however, if you rebuild the 
>> documentation from SVN HEAD and look for "resume" and "suspend" you should 
>> find details for how to do this.
>>
>> With the new code, it is possible to return from the callback and not create 
>> busy waiting.
>>
>> I hope this helps!
>>
>>
>> Best regards,
>>
>> Christian
>>
>> On 10/24/2013 04:02 PM, Adam Homolya wrote:
>>> Dear Christian,
>>>
>>> First of all sorry for disturbing you. At my company, we are planning to 
>>> use libmicrohttpd as a web server frontend for one of our projects. I've 
>>> been evaluating the library for a while and I encountered a serious problem 
>>> (didn't find any solution in the docs).
>>> I would like to stream large amount of data for the clients. The obvious 
>>> solution for that would be creating response from callback 
>>> (MHD_create_response_from_callback). My problem is that libmicrohttpd 
>>> invokes the content reader callback when it wants to send data, and not 
>>> when data is available, so that it could block, which is not an option for 
>>> us. Returning from the callback (marking that no data was read) causes 
>>> "busy wait" like behavior which is even worse. I am looking for some way 
>>> for telling the library not to call the content reader callback until data 
>>> is available for read in a non-blocking manner. Is it possible to work out 
>>> somehow?  Thank you very much.
>>>
>>> Best Regards,
>>>
>>> --
>>> Adam Almos Homolya
>>> Software Engineer
>>> LogMeIn Inc.
>>>