[GNUnet-SVN] r24444 - in libmicrohttpd: . src/daemon

[gnunet]

Author: grothoff
Date: 2012-10-22 13:07:36 +0200 (Mon, 22 Oct 2012)
New Revision: 24444

Modified:
   libmicrohttpd/ChangeLog
   libmicrohttpd/src/daemon/daemon.c
   libmicrohttpd/src/daemon/internal.h
Log:
Trying to fix issue reported by Matthieu:

>>
I face an issue while handling big https posts with libmicrohttpd 0.9.22.
I'm using an "external" select based on MHD_get_fdset and MHD_run.

Let's assume 16K of encrypted data arrive on the socket.
When the data arrives, my select correctly returns and MHD_run is called.

Do_read will first call gnu_tls to decrypt the data.
Apparently gnu_tls reads the full 16K so starting here, from the point of
view of socket/select, there is no more data to read.

Do_read however presents a buffer of only 9K to gnu_tls to retrieve the data
(9K as the result of a try_grow_read_buffer call).
So gnu_tls returns 9K of clear data and keeps the rest 7K in its own buffer
I think.
Do_read handles the 9K and then MHD_run returns;

Since from the socket point of view no data is pending read, select will
then wait until it finally timeout.
Only after this timeout, MHD_run is run again and the remaining 7K are
correctly processed.

I found two dirty workarounds for this issue :
1/ increase the MDH_BUF_INC_SIZE to an arbitrarily large value so it will
always be greater than gnu_tls own buffers
2/ run multiple MHD_run after each select (let's say 10 times)

What do you think of this issue ?
What could be a cleaner way of handling this ?
<<
Approach: detect that we *might* be in this situation because TLS fills
the entire buffer we give to gnuTLS.  If so, set timeout to 0 and read
again.


Modified: libmicrohttpd/ChangeLog
===================================================================
--- libmicrohttpd/ChangeLog     2012-10-22 10:48:29 UTC (rev 24443)
+++ libmicrohttpd/ChangeLog     2012-10-22 11:07:36 UTC (rev 24444)
@@ -1,3 +1,8 @@
+Mon Oct 22 13:05:01 CEST 2012
+       Immediately do a second read if we get a full buffer from
+       TLS as there might be more data in the TLS buffers even if
+       there is no activity on the socket. -CG
+
 Tue Oct 16 01:33:55 CEST 2012
        Consistently use "#ifdef" and "#ifndef" WINDOWS, and not
        sometimes "#if". -CG

Modified: libmicrohttpd/src/daemon/daemon.c
===================================================================
--- libmicrohttpd/src/daemon/daemon.c   2012-10-22 10:48:29 UTC (rev 24443)
+++ libmicrohttpd/src/daemon/daemon.c   2012-10-22 11:07:36 UTC (rev 24444)
@@ -386,6 +386,7 @@
 {
   int res;
 
+  connection->tls_read_ready = MHD_NO;
   res = gnutls_record_recv (connection->tls_session, other, i);
   if ( (GNUTLS_E_AGAIN == res) ||
        (GNUTLS_E_INTERRUPTED == res) )
@@ -401,6 +402,8 @@
       errno = EPIPE;
       return res;
     }
+  if (res == i)
+    connection->tls_read_ready = MHD_YES;
   return res;
 }
 
@@ -606,7 +609,15 @@
          tv.tv_usec = 0;
          tvp = &tv;
        }
-
+#if HTTPS_SUPPORT
+      if (MHD_YES == con->tls_read_ready)
+       {
+         /* do not block (more data may be inside of TLS buffers waiting for 
us) */
+         tv.tv_sec = 0;
+         tv.tv_usec = 0;
+         tvp = &tv;
+       }
+#endif
       if (0 == (con->daemon->options & MHD_USE_POLL))
        {
          /* use select */
@@ -629,7 +640,11 @@
              break;
            }
          /* call appropriate connection handler if necessary */
-         if (FD_ISSET (con->socket_fd, &rs))
+         if ( (FD_ISSET (con->socket_fd, &rs))
+#if HTTPS_SUPPORT
+                  || (MHD_YES == con->tls_read_ready) 
+#endif
+              )
            con->read_handler (con);
          if (FD_ISSET (con->socket_fd, &ws))
            con->write_handler (con);
@@ -660,7 +675,11 @@
 #endif
              break;
            }
-         if (0 != (p[0].revents & POLLIN)) 
+         if ( (0 != (p[0].revents & POLLIN)) 
+#if HTTPS_SUPPORT
+              || (MHD_YES == con->tls_read_ready) 
+#endif
+              )
            con->read_handler (con);        
          if (0 != (p[0].revents & POLLOUT)) 
            con->write_handler (con);        
@@ -1271,6 +1290,14 @@
   have_timeout = MHD_NO;
   for (pos = daemon->connections_head; NULL != pos; pos = pos->next)
     {
+#if HTTPS_SUPPORT
+      if (MHD_YES == pos->tls_read_ready)
+       {
+         earliest_deadline = 0;
+         have_timeout = MHD_YES;
+         break;
+       }
+#endif
       if (0 != pos->connection_timeout) 
        {
          if ( (! have_timeout) ||
@@ -1397,7 +1424,11 @@
           ds = pos->socket_fd;
           if (ds != -1)
             {
-              if (FD_ISSET (ds, &rs))
+              if ( (FD_ISSET (ds, &rs))
+#if HTTPS_SUPPORT
+                  || (MHD_YES == pos->tls_read_ready) 
+#endif
+                  )
                 pos->read_handler (pos);
               if (FD_ISSET (ds, &ws))
                 pos->write_handler (pos);

Modified: libmicrohttpd/src/daemon/internal.h
===================================================================
--- libmicrohttpd/src/daemon/internal.h 2012-10-22 10:48:29 UTC (rev 24443)
+++ libmicrohttpd/src/daemon/internal.h 2012-10-22 11:07:36 UTC (rev 24444)
@@ -750,6 +750,12 @@
    */
   int cipher;
 
+  /**
+   * Could it be that we are ready to read due to TLS buffers
+   * even though the socket is not?
+   */
+  int tls_read_ready;
+
 #endif
 };