Re: [libmicrohttpd] a strange thing when send call returns -1 in send_cls()

[Taehwan Weon]

I am testing reliability of my caching proxy based on MHD, which opens 80 port. 

It means anyone can use it to reach any web server. 

Most of incoming requests seems either incomplete or odd.

(I found a client opened a HTTP session and sent only a part of headers, but no body data.)

1. To avoid spurious EAGAIN, I add a counter into MHD_Connection, which counts continuous errors. If a threshold reaches, ECONNRESET set. EAGAIN seems caused by buffer full on client side. I am not sure what is the client's purpose.

2, To avoid infinite wait to receive remaining part of a client's data, I need timeout mechanism.

    I am not sure where is the best location to handle timeout. MHD or my caching proxy would be the candidate.

------

weon

2014-11-01 4:23 GMT+09:00 Christian Grothoff <address@hidden>:

MHD should tolerate EAGAIN (and just retry), as the kernel can generate
that 'spuriously' (i.e. without good reason). Are we really terminating
the connection on EAGAIN? Is this reproducable? Can you produce a stack
trace for a case where we get an EAGAIN and then close the connection?

If we (as we should) ignore EGAIN and try again, this should be OK, and
might at worst be a kernel-level performance issue (useless context
switching).

Happy hacking!

Christian

On 10/31/2014 07:39 AM, Taehwan Weon wrote:
> Hi,
>
> I applied the final MHD update to my project.
> Still some strange result.
>
> Below is my MHD log generated by my custom send_param_adaptor()
>
> 15:31:58/000 {16138} [ INFO]  message repeated 8042083 times, MHD: socket
> 483 - 12165 bytes tried, but ret = -1, errno = 11
> 15:32:09/000 {16138} [ INFO]  message repeated 8534594 times, MHD: socket
> 483 - 12165 bytes tried, but ret = -1, errno = 11
> 15:32:20/000 {16138} [ INFO]  message repeated 8542227 times, MHD: socket
> 483 - 12165 bytes tried, but ret = -1, errno = 11
> 15:32:31/000 {16138} [ INFO]  message repeated 8546356 times, MHD: socket
> 483 - 12165 bytes tried, but ret = -1, errno = 11
> 15:32:32/889 {16132} [ INFO]  MHD: socket 479 - 385 bytes tried, but ret =
> -1, errno = 104
> 15:32:32/889 {16138} [ INFO]  MHD: socket 483 - 12165 bytes tried, but ret
> = -1, errno = 11
> 15:32:32/889 {16132} [ INFO]  MHD: Failed to send data: Connection reset by
> peer
> (intentionally omitted)
> 15:32:34/648 {16138} [ INFO]  MHD: socket 483 - 12165 bytes tried, but ret
> = -1, errno = 11
> 15:32:34/648 {16132} [ INFO]  MHD: Failed to send data: Connection reset by
> peer
> 15:32:34/648 {16138} [ INFO]  MHD: socket 483 - 12165 bytes tried, but ret
> = -1, errno = 11
> 15:32:45/000 {16138} [ INFO]  message repeated 8033311 times, MHD: socket
> 483 - 12165 bytes tried, but ret = -1, errno = 11
>
> As you see, there are too many erroneous repeats on the socket even in
> error case. (errno is 11 (EAGAIN))
> In such a case, I don't know what to do.
>
>
> Thanks in advance.
>
>
>
> ------
> weon
>
>
> 2014-10-30 0:53 GMT+09:00 Christian Grothoff <address@hidden>:
>
>> On 10/29/2014 08:42 AM, Taehwan Weon wrote:
>>> While testing our web-server (based on MHD 0.9.38), sometimes, I found
>>> heavy cpu load.
>>> It caused by infinite send_cls() calls because send call in send_cls()
>>> returned -1 with errno of 0.
>>>
>>> I read and read again send() manual page, but there is no mention about
>> the
>>> case that send() returns -1 with errno of 0.
>>
>> I agree, that's why our logic doesn't quite expect that. But, nobody
>> said libc/kernel isn't allowed to have bugs ;-).  So my suggestion would
>> be this: I modify MHD to handle this (by closing the connection, don't
>> see what else we can do), and you should please take it up with CentOS,
>> asking them to fix that send() call. (What might be relevant these days:
>> is this using GNU libc or diet libc / ulibc?)
>>
>>> My current environment is:
>>>      OS Kernel :  2.6.32-358.11.5.el6.x86_64 (CentOS)
>>>      MHD options:  MHD_USE_POLL_INTERNALLY |
>> MHD_USE_THREAD_PER_CONNECTION|
>>> MHD_USE_DEBUG;
>>>
>>>
>>> Any help will be highly appreciated.
>>
>> I've patched MHD in SVN 34391, please let us know if that helps (note
>> that IF the kernel does return -1 on send() with no errno, the new logic
>> should simply close the connection).
>>
>> Happy hacking!
>>
>> Christian
>>
>>
>
>

--

원 태환ㅣ 부사장/COO

(주)솔박스

서울시 강남구 역삼동 747-2 해성빌딩 7층

T 02-2182-3600     F 02-2058-2651

M 010-3335-8258  E address@hidden