[libmicrohttpd] new API proposal for HTTP UPGRADE / websockets

[Christian Grothoff]

Dear all,

I've finally made up my mind on a decent API for the "HTTP UPGRADE" 
implementation.  HTTP UPGRADE is used for websockets (and possibly other 
HTTP-extensions) to switch a TCP socket from HTTP-mode to some other, 
application-specific bi-directional protocol.  As mentioned here a few 
months ago, the problem here is to fit this into MHD's execution model. 
Specifically, a first "trivial" API that I proposed had the serious 
disadvantage of not being implementable with HTTPS. So here is my second 
attempt.  I'm specifically trying to achieve the following key design goals:

1) Make the HTTP-UPGRADE API look exactly the same for HTTP and HTTPS.  
If you 'upgrade' an SSL-connection, it should look and feel the same as 
if you upgrade an HTTP connection, the only difference being is that 
your application-protocol is still encrypted with SSL.

2) Make the API work nicely with all of our threading models 
(internal/external/threaded, etc.).

3) Expose limited, but useful low-level TCP capabilities (specifically, 
uncorking) that are useful for high-performance application protocols, 
without exposing the actual socket to the application (remember, we may 
not have a socket, as this may be an HTTPS-connection).

4) Keep it simple (yeah, right).

I've pasted the new definitions from the API below. Nothing has been 
implemented at this point, but comments on the API would be appreciated 
--- especially if it for some reason does NOT fit someone's needs / desires.

Happy hacking!


Christian
///////////////////////////

/**
 * Bits in an event mask that specifies which actions
 * MHD should perform and under which conditions it
 * should call the 'upgrade' callback again.
 */
enum MHD_UpgradeEventMask
{

  /**
   * Never call the handler again; finish sending bytes
   * in the 'write' buffer and then close the socket.
   */
  MHD_UPGRADE_EVENT_TERMINATE = 0,

  /**
   * Call the handler again once there is data ready
   * for reading.
   */
  MHD_UPGRADE_EVENT_READ = 1,

  /**
   * Call the handler again once there is buffer space
   * available for writing.
   */
  MHD_UPGRADE_EVENT_WRITE = 2,

  /**
   * Do not wait on any socket actions, we're waiting on
   * an 'external' event.  Run the function again once
   * the 'select' call returns _without_ this socket even
   * being involved in the select sets (useful in
   * conjunction with the external select loop).
   */
  MHD_UPGRADE_EVENT_EXTERNAL = 4,

  /**
   * Uncork the TCP write buffer (that is, tell the OS to transmit all
   * bytes in the buffer now, and to not use TCP-CORKing).  This is
   * not really an event flag, but an additional request (which MHD
   * may ignore if the platform does not support it).  Note that
   * only returning 'CORK' will *also* cause the socket to be closed!
   */
  MHD_UPGRADE_EVENT_CORK = 8

};


/**
 * Function called after a protocol "upgrade" response was sent
 * successfully and the socket should now be controlled by some
 * protocol other than HTTP.
 *
 * Any data received on the socket will be made available in
 * 'data_in'.  The function should update 'data_in_size' to
 * reflect the number of bytes consumed from 'data_in' (the remaining
 * bytes will be made available in the next call to the handler).
 *
 * Any data that should be transmitted on the socket should be
 * stored in 'data_out'.  '*data_out_size' is initially set to
 * the available buffer space in 'data_out'.  It should be set to
 * the number of bytes stored in 'data_out' (which can be zero).
 *
 * The return value is a BITMASK that indicates how the function
 * intends to interact with the event loop.  It can request to be
 * notified for reading, writing, request to UNCORK the send buffer
 * (which MHD is allowed to ignore, if it is not possible to uncork on
 * the local platform), to wait for the 'external' select loop to
 * trigger another round.  It is also possible to specify "no events"
 * to terminate the connection; in this case, the
 * MHD_RequestCompletedCallback will be called and all resources of
 * the connection will be released.
 *
 * Except when in 'thread-per-connection' mode, implementations
 * of this function should never block (as it will still be called
 * from within the main event loop).
 *
 * @param cls closure
 * @param connection original HTTP connection handle,
 *                   giving the function a last chance
 *                   to inspect the original HTTP request
 * @param con_cls value as set by the last call to the
 *                MHD_AccessHandlerCallback; will afterwards
 *                be also given to the MHD_RequestCompletedCallback
 * @param data_in_size available data for reading, set to data read
 * @param data_in data read from the socket
 * @param data_out_size available buffer for writing, set to bytes
 *                written to 'data_out'
 * @param data_out buffer for sending data via the connection
 * @return desired actions for event handling loop
 */
typedef enum MHD_UpgradeEventMask (*MHD_UpgradeHandler)(void *cls,
                            struct MHD_Connection *connection,
                            void **con_cls,
                            size_t *data_in_size,
                            const char *data_in,
                            size_t *data_out_size,
                            char *data_out);


/**
 * Create a response object that can be used for 101 UPGRADE
 * responses, for example to implement websockets.  After sending the
 * response, control over the data stream is given to the callback (which
 * can then, for example, start some bi-directional communication).
 * If the response is queued for multiple connections, the callback
 * will be called for each connection.  The callback
 * will ONLY be called if the response header was successfully passed
 * to the OS; if there are communication errors before, the usual MHD
 * connection error handling code will be performed.
 *
 * Setting the correct HTTP code (i.e. MHD_HTTP_SWITCHING_PROTOCOLS)
 * and setting correct HTTP headers for the upgrade must be done
 * manually (this way, it is possible to implement most existing
 * WebSocket versions using this API; in fact, this API might be useful
 * for any protocol switch, not just websockets).  Note that
 * draft-ietf-hybi-thewebsocketprotocol-00 cannot be implemented this
 * way as the header "HTTP/1.1 101 WebSocket Protocol Handshake"
 * cannot be generated; instead, MHD will always produce "HTTP/1.1 101
 * Switching Protocols" (if the response code 101 is used).
 *
 * As usual, the response object can be extended with header
 * information and then be used any number of times (as long as the
 * header information is not connection-specific).
 *
 * @param upgrade_handler function to call with the 'upgraded' socket
 * @param upgrade_handler_cls closure for 'upgrade_handler'
 * @return NULL on error (i.e. invalid arguments, out of memory)
 */
struct MHD_Response *
MHD_create_response_for_upgrade (MHD_UpgradeHandler upgrade_handler,
                 void *upgrade_handler_cls);