Re: [Paperclips-discuss] Http performance

[Christopher K. St. John]

Nic Ferrier wrote:
> 
> Part of chris' problem is probably that Paperclips is
> not setting up the socketserver very wisely right now.
> 

 Turns out that's not the thing. I'm not 100% sure of all
of this, so you'll have to bear with me as I go step by
step:

 - I added code to HTTPConnection.handleConnection()
 because I wanted to see where the hang was occurring.

 - Turns out the hang was in readHeaders(), so I added
 more debugging output and it's hanging the second time
 round because it thinks the connection is persistent.

 - It's a good assumption: httperf sends a 1.1 version
 and doesn't send a Connection:close header.

 - But: httperf closes the connection anyway. This is
 legal. It violates a SHOULD for clients, but that's
 ok, since servers MUST handle it properly[1]

 - It that really what's going on? ethereal[2]/tcpdump
 sez yes:

     0 echidna  -> duckbill  SYN
     1 duckbill -> echidna   SYN, ACK
     2 echidna  -> duckbill  ACK
     3 echidna  -> duckbill  GET /hp2.html HTTP/1.1
     4 duckbill -> echidna   ACK
     5 duckbill -> echidna   HTTP/1.1 200 Ok
     6 echidna  -> duckbill  ACK
     7 duckbill -> echidna   <headers+body>
     8 echidna  -> duckbill  FIN, ACK
     9 duckbill -> echidna   ACK
 
 Tsk tsk, response first-line in one packet, headers
 and body in another. But that's legal. What's bad
 is that sure enough, httperf is FIN'ing, but pclips
 isn't FIN'ing back.

 - Which brings us to HTTPInStream. timeout() is a
 very scary method. This is where I'm not 100% sure,
 but from the api docs, available() is free to return
 0 even when the stream is closed. And sleeping for
 100ms during a read is just plain evil.

 - If you really gotta time out during a read, then
 the only way to do it is Socket.setSoTimeout(). It's
 guaranteed to work cross-platform, the call has been
 available since 1.1.*, and it's not weird (== people
 will 'get it' when they see the code)

 - Optionally, you can have a reaper-thread that runs
 around and closes the socket out from under readLine().
 This is 'Sun approved' and also guaranteed to work.

 - What to do? setSoTimeout probably fits into the 
 current code better than a reaper thread, so put
 that in, take HTTPInStream.timeout() out.

 - Which I did. Well, I didn't put in a setSoTimeout,
 but I commented out timeout() and it worked. Sort
 of. Still random pauses, some quite long, but it
 didn't every (quite) fully hang. Not sure why.


[1] rfc2616, 8.1.4 Practical Considerations

     ... A client, server, or proxy MAY close the
     transport connection at any time ...  This
     means that clients, servers, and proxies MUST
     be able to recover from asynchronous close
     events. ...

[2] Ethereal - 'Sniffing the glue that holds the
    Internet Together' http://www.ethereal.com
    Yes, real men use tcpdump, but ethereal is
    so pretty. And it knows about HTTP.


-- 
Christopher St. John address@hidden
DistribuTopia http://www.distributopia.com