[libmicrohttpd] Re: libmicrohttp upload issue

[John Muth]

Hi Christian,

I'm pretty sure what's happening is the read buffer keeps getting shrunk as a result of reading headers for each request. It is indeed the server issuing small reads to the socket.

I added some messages inside libmicrohttpd to try to figure out what's going on. I added messages to every place where connection->read_buffer_size is changed. What I found is the read buffer shrinks for every header that gets read in. For example, from a message at the end of connection.c:get_next_header_line():

--- 111 set read_buffer_size=2048
--- recv: req len: 2048 got:111 bufsz:2048 off:0
--- 333 set read_buffer_size=1974 GET /stor/stat/vol=1/fsid=1/fnum=2/chunk=3/version=4/ticket=TBD HTTP/1.1
--- 333 set read_buffer_size=1952 Host: localhost:9000
--- 333 set read_buffer_size=1939 Accept: */*
--- 333 set read_buffer_size=1937
--- 555 pool_reset size=1937
--- recv: req len: 1937 got:179 bufsz:1937 off:0
--- 333 set read_buffer_size=1840 PUT /stor/write/offset=0/length=65536/vol=1/fsid=1/fnum=2/chunk=3/version=4/ticket=TBD HTTP/1.1
--- 333 set read_buffer_size=1818 Host: localhost:9000
--- 333 set read_buffer_size=1805 Accept: */*
--- 333 set read_buffer_size=1782 Content-Length: 65536
--- 333 set read_buffer_size=1760 Expect: 100-continue
--- 333 set read_buffer_size=1758
--- recv: req len: 1758 got:1758 bufsz:1758 off:0
--- recv: req len: 1758 got:1758 bufsz:1758 off:0

Later I see:
--- 555 pool_reset size=1758
--- recv: req len: 1758 got:183 bufsz:1758 off:0
--- 333 set read_buffer_size=1657 PUT /stor/write/offset=65536/length=65536/vol=1/fsid=1/fnum=2/chunk=3/version=4/ticket=TBD HTTP/1.1
--- 333 set read_buffer_size=1635 Host: localhost:9000
--- 333 set read_buffer_size=1622 Accept: */*
--- 333 set read_buffer_size=1599 Content-Length: 65536
--- 333 set read_buffer_size=1577 Expect: 100-continue
--- 333 set read_buffer_size=1575
--- recv: req len: 1575 got:1575 bufsz:1575 off:0
--- recv: req len: 1575 got:1575 bufsz:1575 off:0
--- recv: req len: 1575 got:1575 bufsz:1575 off:0

After that I see:

--- 555 pool_reset size=1575
--- recv: req len: 1575 got:184 bufsz:1575 off:0
--- 333 set read_buffer_size=1473 PUT /stor/write/offset=131072/length=65536/vol=1/fsid=1/fnum=2/chunk=3/version=4/ticket=TBD HTTP/1.1
--- 333 set read_buffer_size=1451 Host: localhost:9000
--- 333 set read_buffer_size=1438 Accept: */*
--- 333 set read_buffer_size=1415 Content-Length: 65536
--- 333 set read_buffer_size=1393 Expect: 100-continue
--- 333 set read_buffer_size=1391
--- recv: req len: 1391 got:1391 bufsz:1391 off:0
--- recv: req len: 1391 got:1391 bufsz:1391 off:0


And so on until libmicrohttpd is issuing very small reads to to socket.

Is there anything I can do to help you with this? I'd really like to use libmicrohttpd for this project.

-John

On 5/15/09, Christian Grothoff <address@hidden> wrote:

Simple answer: no ;-).  I'd try strace first to see if either the *client* is
sending in strange increments or if the server is reading in small increments
(which it should not be).


Christian


On Thursday 14 May 2009 11:54:10 pm John Muth wrote:
> Hi,
>
> I'm doing a project with libmicrohttp 0.4.1 and I'm seeing some weird
> behavior when I upload data (I'm using libcurl 7.19.4 to drive the client
> side to issue a PUT and push the data up). I'm doing 64K transfers.
>
> On the server side, I'm seeing my access handler called with upload data
> but in very small increments. For example, the first transfer will be a
> series of 1758 byte uploads. The next transfer will be a series of 1575
> bytes transfers. The 3rd will be 1391 byte transfers. And so on until I see
> a series of 2 byte transfers.
>
> Have you seen anything like this before?
>
> -John Muth
> Parascale