[libmicrohttpd] Some issues with MHD

[Eugenio Perez]

Hi All.

First of all, congrats for this excellent library! I think it does
have a clear interface and it's very usable.

However, I have found two issues. I don't know if I'm using bad the
library, so please correct me if I'm wrong.

I'm trying to send, using one connection, a lot of small (~2K) POST
messages (in the real case, JSON text). If I use not chunked encoding
messages (i.e., specifying Content-Length header), all is OK.

However, I've found that if I try to send the messages in chunks, MHD
drops the connection, and the client needs to reconnect again. Testing
in deep, I've found that:

* If I increase MHD_OPTION_CONNECTION_MEMORY_LIMIT, the connections
are dropped slower (i.e., the same connection last longer).
* If I send smaller messages that the sent in http-post.c, MHD does not drop.
* If I change URL length, drops rate changes:
  * ./http-post "http://10.0.70.175:8080/aaaaaaaaaaaaaaaaaaaaaaaaaaaa";
causes drops
  * ./http-post
"http://10.0.70.175:8080/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; no drops
* If I apply the patch attached, MHD does not drop my connection
anymore. However, I don't know if it affects some other MHD internals.

All of this happens with MHD 0.9.44. I've seen that you just announced
0.9.45. However, if I update is even worse: connections are being
dropped almost every packet or 2 packet! If I bisect, the commit that
introduce the issue is r36568 (with message "-fix assertion failure
from race on shutdown and buffer shrinkage with pipelining").

I attach client and server I'm using to test situation. In order to
detect reconnection, I just use:

tcpdump -nni any "port 8080 and tcp[tcpflags] & (tcp-syn) != 0"

Where 8080 is the post I'm using to testing.

Am I using the bad options? Is the patch valid to fix the issue? if
not, what could I do to keep debugging this?

Thanks and regards.

http-post.c
Description: Text Data

minimal_example.c
Description: Text Data

MHD-chunked-drops.patch
Description: Text Data