Re: [libmicrohttpd] new API proposal for HTTP UPGRADE / websockets

[Simon Newton]

This looks good to me.


On Sun, Nov 25, 2012 at 5:11 AM, Christian Grothoff
<address@hidden> wrote:
> Dear all,
>
> I've finally made up my mind on a decent API for the "HTTP UPGRADE"
> implementation.  HTTP UPGRADE is used for websockets (and possibly other
> HTTP-extensions) to switch a TCP socket from HTTP-mode to some other,
> application-specific bi-directional protocol.  As mentioned here a few
> months ago, the problem here is to fit this into MHD's execution model.
> Specifically, a first "trivial" API that I proposed had the serious
> disadvantage of not being implementable with HTTPS. So here is my second
> attempt.  I'm specifically trying to achieve the following key design goals:
>
> 1) Make the HTTP-UPGRADE API look exactly the same for HTTP and HTTPS.  If
> you 'upgrade' an SSL-connection, it should look and feel the same as if you
> upgrade an HTTP connection, the only difference being is that your
> application-protocol is still encrypted with SSL.
>
> 2) Make the API work nicely with all of our threading models
> (internal/external/threaded, etc.).
>
> 3) Expose limited, but useful low-level TCP capabilities (specifically,
> uncorking) that are useful for high-performance application protocols,
> without exposing the actual socket to the application (remember, we may not
> have a socket, as this may be an HTTPS-connection).
>
> 4) Keep it simple (yeah, right).
>
> I've pasted the new definitions from the API below. Nothing has been
> implemented at this point, but comments on the API would be appreciated ---
> especially if it for some reason does NOT fit someone's needs / desires.
>
> Happy hacking!
>
>
> Christian
> ///////////////////////////
>
> /**
>  * Bits in an event mask that specifies which actions
>  * MHD should perform and under which conditions it
>  * should call the 'upgrade' callback again.
>  */
> enum MHD_UpgradeEventMask
> {
>
>   /**
>    * Never call the handler again; finish sending bytes
>    * in the 'write' buffer and then close the socket.
>    */
>   MHD_UPGRADE_EVENT_TERMINATE = 0,
>
>   /**
>    * Call the handler again once there is data ready
>    * for reading.
>    */
>   MHD_UPGRADE_EVENT_READ = 1,
>
>   /**
>    * Call the handler again once there is buffer space
>    * available for writing.
>    */
>   MHD_UPGRADE_EVENT_WRITE = 2,
>
>   /**
>    * Do not wait on any socket actions, we're waiting on
>    * an 'external' event.  Run the function again once
>    * the 'select' call returns _without_ this socket even
>    * being involved in the select sets (useful in
>    * conjunction with the external select loop).
>    */
>   MHD_UPGRADE_EVENT_EXTERNAL = 4,
>
>   /**
>    * Uncork the TCP write buffer (that is, tell the OS to transmit all
>    * bytes in the buffer now, and to not use TCP-CORKing).  This is
>    * not really an event flag, but an additional request (which MHD
>    * may ignore if the platform does not support it).  Note that
>    * only returning 'CORK' will *also* cause the socket to be closed!
>    */
>   MHD_UPGRADE_EVENT_CORK = 8
>
> };
>
>
> /**
>  * Function called after a protocol "upgrade" response was sent
>  * successfully and the socket should now be controlled by some
>  * protocol other than HTTP.
>  *
>  * Any data received on the socket will be made available in
>  * 'data_in'.  The function should update 'data_in_size' to
>  * reflect the number of bytes consumed from 'data_in' (the remaining
>  * bytes will be made available in the next call to the handler).
>  *
>  * Any data that should be transmitted on the socket should be
>  * stored in 'data_out'.  '*data_out_size' is initially set to
>  * the available buffer space in 'data_out'.  It should be set to
>  * the number of bytes stored in 'data_out' (which can be zero).
>  *
>  * The return value is a BITMASK that indicates how the function
>  * intends to interact with the event loop.  It can request to be
>  * notified for reading, writing, request to UNCORK the send buffer
>  * (which MHD is allowed to ignore, if it is not possible to uncork on
>  * the local platform), to wait for the 'external' select loop to
>  * trigger another round.  It is also possible to specify "no events"
>  * to terminate the connection; in this case, the
>  * MHD_RequestCompletedCallback will be called and all resources of
>  * the connection will be released.
>  *
>  * Except when in 'thread-per-connection' mode, implementations
>  * of this function should never block (as it will still be called
>  * from within the main event loop).
>  *
>  * @param cls closure
>  * @param connection original HTTP connection handle,
>  *                   giving the function a last chance
>  *                   to inspect the original HTTP request
>  * @param con_cls value as set by the last call to the
>  *                MHD_AccessHandlerCallback; will afterwards
>  *                be also given to the MHD_RequestCompletedCallback
>  * @param data_in_size available data for reading, set to data read
>  * @param data_in data read from the socket
>  * @param data_out_size available buffer for writing, set to bytes
>  *                written to 'data_out'
>  * @param data_out buffer for sending data via the connection
>  * @return desired actions for event handling loop
>  */
> typedef enum MHD_UpgradeEventMask (*MHD_UpgradeHandler)(void *cls,
>                             struct MHD_Connection *connection,
>                             void **con_cls,
>                             size_t *data_in_size,
>                             const char *data_in,
>                             size_t *data_out_size,
>                             char *data_out);
>
>
> /**
>  * Create a response object that can be used for 101 UPGRADE
>  * responses, for example to implement websockets.  After sending the
>  * response, control over the data stream is given to the callback (which
>  * can then, for example, start some bi-directional communication).
>  * If the response is queued for multiple connections, the callback
>  * will be called for each connection.  The callback
>  * will ONLY be called if the response header was successfully passed
>  * to the OS; if there are communication errors before, the usual MHD
>  * connection error handling code will be performed.
>  *
>  * Setting the correct HTTP code (i.e. MHD_HTTP_SWITCHING_PROTOCOLS)
>  * and setting correct HTTP headers for the upgrade must be done
>  * manually (this way, it is possible to implement most existing
>  * WebSocket versions using this API; in fact, this API might be useful
>  * for any protocol switch, not just websockets).  Note that
>  * draft-ietf-hybi-thewebsocketprotocol-00 cannot be implemented this
>  * way as the header "HTTP/1.1 101 WebSocket Protocol Handshake"
>  * cannot be generated; instead, MHD will always produce "HTTP/1.1 101
>  * Switching Protocols" (if the response code 101 is used).
>  *
>  * As usual, the response object can be extended with header
>  * information and then be used any number of times (as long as the
>  * header information is not connection-specific).
>  *
>  * @param upgrade_handler function to call with the 'upgraded' socket
>  * @param upgrade_handler_cls closure for 'upgrade_handler'
>  * @return NULL on error (i.e. invalid arguments, out of memory)
>  */
> struct MHD_Response *
> MHD_create_response_for_upgrade (MHD_UpgradeHandler upgrade_handler,
>                  void *upgrade_handler_cls);
>