Re: [libmicrohttpd] Problem send large trunk of data over upgraded connection with epoll and https enabled

[Christian Grothoff]

Dear Viet,

I've now managed to fix the issue in Git master.

Basically, we incorrectly decided that the read-buffer had been drained
when gnutls_record_recv() returned a partial read. However, gnutls may
return a partial read even if the read buffer has not been drained.

Because we use edge polling, we must drain the buffer, which is only the
case if gnutls_record_recv() returns 0.  I've fixed this and a few
possibly related corner issues (with gnutls_record_uncork) in Git master.

Happy hacking!

Christian

On 9/18/20 5:26 AM, Nguyen Xuan Viet wrote:
> Dear Christian,
> 
> I'm sorry but this bug still happen on latest version of this library.
> The "test_upgrade_large_tls" still hangs with very large data send over
> the upgraded connection.
> We can reproduce this issue by increase "LARGE_STRING" and/or
> "LARGE_REPLY_STRING" size to a huge string, for example duplicate
> current "Hello..." string to 41 lines or more.
> 
> This issue only happens with EPOLL + HTTPS, for other cases, it works
> normally.
> Could you please check this issue for me?
> 
> Thanks,
> Viet
> 
> On 7/30/19 11:00 PM, Việt Nguyễn Xuân wrote:
>> Dear Christian,
>>
>> Thanks for your reply.
>> I'm very happy to hear your information. I will continue to work with your 
>> latest source in the future.
>>
>> Best regards,
>> Viet
>> On 30/07/2019 01:04, Christian Grothoff wrote:
>>> Dear Viet,
>>>
>>> Thanks for the additional details.  With your report, I was able to
>>> reproduce the issue (test_upgrade_large.c added) and fix it in
>>> bafd078984be5f846287107824eca99e6c6c42cc.
>>>
>>> Happy hacking!
>>>
>>> Christian
>>>
>>> On 6/24/19 7:16 PM, Việt Nguyễn Xuân wrote:
>>>> Hi all,
>>>>
>>>> Sorry, I cannot reply your email, so I reply it here.
>>>> I'm sorry, for the moment I cannot provide the code for you. But you can
>>>> reproduce the issue by modify the src/microhttpd/test_upgrade.c, set the
>>>> connection memory limit to a very low value, for example 512, this
>>>> change should be made for HTTPS section. In the function "run_usock" and
>>>> "run_usock_client", please replace the test string "Hello World" by a
>>>> real larger string with length about 160 - 170 bytes. After that do a
>>>> make check and you will see the test_upgrade_tls test hang forever.
>>>>
>>>> Thanks,
>>>> Best regards,
>>>> Viet
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Việt Nguyễn Xuân
>>>> *Sent:* Sunday, June 23, 2019 11:28 AM
>>>> *To:* libmicrohttpd@gnu.org
>>>> *Subject:* [libmicrohttpd]Problem send large trunk of data over upgraded
>>>> connection with epoll and https enabled
>>>>  
>>>> Hi All,
>>>>
>>>> I'm using libmicrohttpd to implement a WebSocket server based on HTTP
>>>> Upgrade connection.
>>>> I'm using latest git revision of this library for my work.
>>>> Everything works well for small message passes over upgraded connection,
>>>> I tested with Chrome and Firefox browsers and they are all Ok.
>>>> But I have a trouble when sending large trunk of message to the server,
>>>> if the size of message is larger than 1/3 connection buffer size, it's
>>>> very often that this message will be queued for next send or is not sent
>>>> at all. For smaller messages, they will be sent and received immediately.
>>>>
>>>> This issue  only happens when I enable HTTPs combine with EPOLL internal
>>>> thread, for other combination it seems that this issue does not happen.
>>>> For example I can send large message normally with HTTPS + POLL internal
>>>> thread or with HTTP + EPOLL internal thread.
>>>>
>>>> Could you please check this issue for me?
>>>> Thanks a lot.
>>>>
>>>> Viet

signature.asc
Description: OpenPGP digital signature