lynx-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

lynx-dev dev.16 patch 3 - HTMIME.c, HTTP.c redirection (long)


From: Klaus Weide
Subject: lynx-dev dev.16 patch 3 - HTMIME.c, HTTP.c redirection (long)
Date: Sat, 4 Dec 1999 09:10:06 -0600 (CST)

* Don't write raw escape sequences to trace output for lynx-keymaps if
  not using trace log file.
* Protect echo() in stop_curses, it could crash if stop_curses gets called
  without preceding start_curses.
* Save size for regular files in anchor structure (we did the stat() anyway),
  it shows up on INFO page as Content-Length.  Also use it in partial display
  mode to prevent the last call to HTDisplayPartial from HTFileCopy, since a
  call to HText_pageDisplay will follow immediately.  (But note that nothing
  important depends on the correct size; should it be wrong, we lose at most
  one partial display screen update.  An equivalent suppression of the last
  partial update for network protocols is not recommended, since (a) the size
  is more unreliable and (b) the socket FIN may get delayed by the network.)
* Split HTMIME_put_character into three functions.
* Made HTMIME.c handle folded header lines correctly (long-standing bug,
  reported in February by Devid Coles).

[ reference: http://www.flora.org/lynx-dev/html/month0299/msg00135.html ]

* Changed how HTTP redirection messages are handled.  Don't read full message
  and parse buffer for some header fields in HTLoadHTTP.  Use common HTMIME.c
  header parsing instead.  Functions added/modified in HTMIME.c for storing
  away the Location URL (if requested and found) and for stopping after the
  headers.  New private MIME types "message/x-http-redirection" and
  "www/debug" to control this via HTStreamStack.  Basic approach taken from
  newer W3C libwww.  Benefits:
  - Removes actual bug that was reported: "Content-Location:" was falsely
    recognized as "Location:". (long-standing bug)

[ reference: http://www.flora.org/lynx-dev/html/month1199/msg00370.html ]

  - Removes potential other bugs caused by inferior header parsing in HTTP.c
    (including Set-Cookie/Set-Cookie2).
  - Should more header fields be needed in the future (also) for redirection
    messages, we now get them automatically instead of having to add extra
    code.
  - Trace output now shows the same kind of info as for other responses.
    In particular, Set-Cookie/Set-Cookie2 header fields will normally be
    visible.
  - Statusline progress messages now actually show the HTTP response line
    as intended, and it is recorded for LYNXMESSAGES:. (Long-standing error,
    previously what was shown could be an empty string, or arbitrary parts
    of the response, including HTML markup from the message body.)
  - Adding redirection support for lynxcgi would now be simpler (although
    currently not done).
* HTErrorStream(), similar to W3C libwww, used in some cases for the above.

Index: 2.30/WWW/Library/Implementation/HTAccess.c
--- 2.30/WWW/Library/Implementation/HTAccess.c Wed, 06 Oct 1999 14:48:20 -0500
+++ 2.30(w)/WWW/Library/Implementation/HTAccess.c Sat, 04 Dec 1999 01:59:20 
-0600
@@ -971,6 +971,13 @@
        **  So, instead, we'll go all the way back to the top of getfile
        **  in LYGetFile.c when the status is HT_REDIRECTING.  This may
        **  seem bizarre, but it works like a charm! - FM
+       **
+       **  Actually, the location header for redirections is now again
+       **  picked up in HTMIME.c.  But that's an internal matter between
+       **  HTTP.c and HTMIME.c, is still under control of HTLoadHTTP for
+       **  http URLs, is done in a way that doesn't load the redirection
+       **  response's body (except when wanted as an error fallback), and
+       **  thus need not concern us here. - kw 1999-12-02
        */
        CTRACE((tfp, "HTAccess: '%s' is a redirection URL.\n",
                    address_to_load));
Index: 2.30/WWW/Library/Implementation/HTTP.c
--- 2.30/WWW/Library/Implementation/HTTP.c Thu, 25 Nov 1999 07:30:47 -0600
+++ 2.30(w)/WWW/Library/Implementation/HTTP.c Sat, 04 Dec 1999 01:59:26 -0600
@@ -295,7 +295,8 @@
   BOOL auth_proxy = NO;        /* Generate a proxy authorization. - AJL */
 
   int length, rawlength, rv;
-  BOOL doing_redirect, already_retrying = FALSE, bad_location = FALSE;
+  int server_status;
+  BOOL doing_redirect, already_retrying = FALSE;
   int len = 0;
 
   void * handle = NULL;
@@ -993,7 +994,6 @@
   {
     int fields;
     char server_version[VERSION_LENGTH+1];
-    int server_status;
 
     server_version[0] = 0;
 
@@ -1278,8 +1278,6 @@
             *  previous document. - FM
             */
            {
-             char *cp;
-
              if ((dump_output_immediately || traversal) &&
                  do_post &&
                  server_status != 303 &&
@@ -1298,36 +1296,8 @@
                  goto clean_up;
              }
 
-             /*
-              *  Get the rest of the headers and data, if
-              *  any, and then close the connection. - FM
-              */
-             while ((status = HTTP_NETREAD(s, line_buffer,
-                                           (INIT_LINE_SIZE - 1),
-                                           handle)) > 0) {
-#ifdef    NOT_ASCII  /* S/390 -- gil -- 0581 */
-             {   char *p;
-
-                 for ( p = line_buffer; p < line_buffer + status; p++ )
-                     *p = FROMASCII(*p);
-             }
-#endif /* NOT_ASCII */
-                 line_buffer[status] = '\0';
-                 StrAllocCat(line_kept_clean, line_buffer);
-             }
-             HTTP_NETCLOSE(s, handle);
-             if (status == HT_INTERRUPTED) {
-                 /*
-                  *  Impatient user. - FM
-                  */
-                 CTRACE((tfp, "HTTP: Interrupted followup read.\n"));
-                 _HTProgress (CONNECTION_INTERRUPTED);
-                 status = HT_INTERRUPTED;
-                 goto clean_up;
-             }
-             doing_redirect = TRUE;
+             HTProgress(line_buffer);
              if (server_status == 301) { /* Moved Permanently */
-                 HTProgress(line_buffer);
                  if (do_post) {
                      /*
                       *  Don't make the redirection permanent
@@ -1340,341 +1310,8 @@
                      permanent_redirection = TRUE;
                  }
              }
+             doing_redirect = TRUE;
 
-             /*
-             **  Look for "Set-Cookie:" and "Set-Cookie2:" headers. - FM
-             */
-             if (LYSetCookies == TRUE) {
-                 char *value = NULL;
-                 char *SetCookie = NULL;
-                 char *SetCookie2 = NULL;
-                 cp = line_kept_clean;
-                 while (*cp) {
-                     /*
-                     **  Assume a CRLF pair terminates
-                     **  the header section. - FM
-                     */
-                     if (*cp == CR) {
-                         if (*(cp+1) == LF &&
-                             *(cp+2) == CR && *(cp+3) == LF) {
-                             break;
-                         }
-                     }
-                     if (TOUPPER(*cp) != 'S') {
-                         cp++;
-                     } else if (!strncasecomp(cp, "Set-Cookie:", 11))  {
-                         char *cp1 = NULL, *cp2 = NULL;
-                         cp += 11;
-Cookie_continuation:
-                         /*
-                          *  Trim leading spaces. - FM
-                          */
-                         while (isspace((unsigned char)*cp))
-                             cp++;
-                         /*
-                         **  Accept CRLF, LF, or CR as end of line. - FM
-                         */
-                         if (((cp1 = strchr(cp, LF)) != NULL) ||
-                             (cp2 = strchr(cp, CR)) != NULL) {
-                             if (*cp1) {
-                                 *cp1 = '\0';
-                                 if ((cp2 = strchr(cp, CR)) != NULL)
-                                     *cp2 = '\0';
-                             } else {
-                                 *cp2 = '\0';
-                             }
-                         }
-                         if (*cp == '\0') {
-                             if (cp1)
-                                 *cp1 = LF;
-                             if (cp2)
-                                 *cp2 = CR;
-                             if (value != NULL) {
-                                 HTMIME_TrimDoubleQuotes(value);
-                                 if (SetCookie == NULL) {
-                                     StrAllocCopy(SetCookie, value);
-                                 } else {
-                                     StrAllocCat(SetCookie, ", ");
-                                     StrAllocCat(SetCookie, value);
-                                 }
-                                 FREE(value);
-                             }
-                             break;
-                         }
-                         StrAllocCat(value, cp);
-                         cp += strlen(cp);
-                         if (cp1) {
-                             *cp1 = LF;
-                             cp1 = NULL;
-                         }
-                         if (cp2) {
-                             *cp2 = CR;
-                             cp2 = NULL;
-                         }
-                         cp1 = cp;
-                         if (*cp1 == CR)
-                            cp1++;
-                         if (*cp1 == LF)
-                            cp1++;
-                         if (*cp1 == ' ' || *cp1 == '\t') {
-                             StrAllocCat(value, " ");
-                             cp = cp1;
-                             cp++;
-                             cp1 = NULL;
-                             goto Cookie_continuation;
-                         }
-                         HTMIME_TrimDoubleQuotes(value);
-                         if (SetCookie == NULL) {
-                             StrAllocCopy(SetCookie, value);
-                         } else {
-                             StrAllocCat(SetCookie, ", ");
-                             StrAllocCat(SetCookie, value);
-                         }
-                         FREE(value);
-                     } else if (!strncasecomp(cp, "Set-Cookie2:", 12))  {
-                         char *cp1 = NULL, *cp2 = NULL;
-                         cp += 12;
-Cookie2_continuation:
-                         /*
-                          *  Trim leading spaces. - FM
-                          */
-                         while (isspace((unsigned char)*cp))
-                             cp++;
-                         /*
-                         **  Accept CRLF, LF, or CR as end of line. - FM
-                         */
-                         if (((cp1 = strchr(cp, LF)) != NULL) ||
-                             (cp2 = strchr(cp, CR)) != NULL) {
-                             if (*cp1) {
-                                 *cp1 = '\0';
-                                 if ((cp2 = strchr(cp, CR)) != NULL)
-                                     *cp2 = '\0';
-                             } else {
-                                 *cp2 = '\0';
-                             }
-                         }
-                         if (*cp == '\0') {
-                             if (cp1)
-                                 *cp1 = LF;
-                             if (cp2)
-                                 *cp2 = CR;
-                             if (value != NULL) {
-                                 HTMIME_TrimDoubleQuotes(value);
-                                 if (SetCookie2 == NULL) {
-                                     StrAllocCopy(SetCookie2, value);
-                                 } else {
-                                     StrAllocCat(SetCookie2, ", ");
-                                     StrAllocCat(SetCookie2, value);
-                                 }
-                                 FREE(value);
-                             }
-                             break;
-                         }
-                         StrAllocCat(value, cp);
-                         cp += strlen(cp);
-                         if (cp1) {
-                             *cp1 = LF;
-                             cp1 = NULL;
-                         }
-                         if (cp2) {
-                             *cp2 = CR;
-                             cp2 = NULL;
-                         }
-                         cp1 = cp;
-                         if (*cp1 == CR)
-                            cp1++;
-                         if (*cp1 == LF)
-                            cp1++;
-                         if (*cp1 == ' ' || *cp1 == '\t') {
-                             StrAllocCat(value, " ");
-                             cp = cp1;
-                             cp++;
-                             cp1 = NULL;
-                             goto Cookie2_continuation;
-                         }
-                         HTMIME_TrimDoubleQuotes(value);
-                         if (SetCookie2 == NULL) {
-                             StrAllocCopy(SetCookie2, value);
-                         } else {
-                             StrAllocCat(SetCookie2, ", ");
-                             StrAllocCat(SetCookie2, value);
-                         }
-                         FREE(value);
-                     } else {
-                         cp++;
-                     }
-                 }
-                 FREE(value);
-                 if (SetCookie != NULL || SetCookie2 != NULL) {
-                     LYSetCookie(SetCookie, SetCookie2, anAnchor->address);
-                     FREE(SetCookie);
-                     FREE(SetCookie2);
-                 }
-             }
-
-             /*
-              *  Look for the "Location:" in the headers. - FM
-              */
-             cp = line_kept_clean;
-             while (*cp) {
-               if (TOUPPER(*cp) != 'L') {
-                   cp++;
-               } else if (!strncasecomp(cp, "Location:", 9)) {
-                   char *cp1 = NULL, *cp2 = NULL;
-                   cp += 9;
-                   /*
-                    *  Trim leading spaces. - FM
-                    */
-                   while (isspace((unsigned char)*cp))
-                       cp++;
-                   /*
-                    *  Accept CRLF, LF, or CR as end of header. - FM
-                    */
-                   if (((cp1 = strchr(cp, LF)) != NULL) ||
-                       (cp2 = strchr(cp, CR)) != NULL) {
-                       if (*cp1) {
-                           *cp1 = '\0';
-                           if ((cp2 = strchr(cp, CR)) != NULL)
-                               *cp2 = '\0';
-                       } else {
-                           *cp2 = '\0';
-                       }
-                       /*
-                        *  Load the new URL into redirecting_url,
-                        *  and make sure it's not zero-length. - FM
-                        */
-                       StrAllocCopy(redirecting_url, cp);
-                       HTMIME_TrimDoubleQuotes(redirecting_url);
-                       if (*redirecting_url == '\0') {
-                           /*
-                            *  The "Location:" value is zero-length, and
-                            *  thus is probably something in the body, so
-                            *  we'll show the user what was returned. - FM
-                            */
-                           CTRACE((tfp, "HTTP: 'Location:' is 
zero-length!\n"));
-                           if (cp1)
-                               *cp1 = LF;
-                           if (cp2)
-                               *cp2 = CR;
-                           bad_location = TRUE;
-                           FREE(redirecting_url);
-                           HTAlert(
-                              gettext("Got redirection with a bad Location 
header."));
-                           HTProgress(line_buffer);
-                           break;
-                       }
-
-                       /*
-                        *  Set up for checking redirecting_url in
-                        *  LYGetFile.c for restrictions before we
-                        *  seek the document at that Location. - FM
-                        */
-                       HTProgress(line_buffer);
-                       CTRACE((tfp, "HTTP: Picked up location '%s'\n",
-                                   redirecting_url));
-                       if (cp1)
-                           *cp1 = LF;
-                       if (cp2)
-                           *cp2 = CR;
-                       if (server_status == 305) { /* Use Proxy */
-                           /*
-                            *  Make sure the proxy field ends with
-                            *  a slash. - FM
-                            */
-                           if (redirecting_url[strlen(redirecting_url)-1]
-                               != '/')
-                               StrAllocCat(redirecting_url, "/");
-                           /*
-                            *  Append our URL. - FM
-                            */
-                           StrAllocCat(redirecting_url, anAnchor->address);
-                           CTRACE((tfp, "HTTP: Proxy URL is '%s'\n",
-                                       redirecting_url));
-                       }
-                       if (!do_post ||
-                           server_status == 303 ||
-                           server_status == 302) {
-                           /*
-                            *  We don't have POST content (nor support PUT
-                            *  or DELETE), or the status is "See Other"  or
-                            *  "General Redirection" and we can convert to
-                            *  GET, so go back and check out the new URL. - FM
-                            */
-                           status = HT_REDIRECTING;
-                           goto clean_up;
-                       }
-                       /*
-                        *  Make sure the user wants to redirect
-                        *  the POST content, or treat as GET - FM & DK
-                        */
-                       switch (HTConfirmPostRedirect(redirecting_url,
-                                                     server_status)) {
-                           /*
-                            *  User failed to confirm.
-                            *  Abort the fetch.
-                            */
-                           case 0:
-                               doing_redirect = FALSE;
-                               FREE(redirecting_url);
-                               status = HT_NO_DATA;
-                               goto clean_up;
-
-                           /*
-                            *  User wants to treat as GET with no content.
-                            *  Go back to check out the URL.
-                            */
-                           case 303:
-                               status = HT_REDIRECTING;
-                               goto clean_up;
-
-                           /*
-                            *  Set the flag to retain the POST
-                            *  content and go back to check out
-                            *  the URL. - FM
-                            */
-                           default:
-                               status = HT_REDIRECTING;
-                               redirect_post_content = TRUE;
-                               goto clean_up;
-                       }
-                   }
-                   break;
-               } else {
-                   /*
-                    *  Keep looking for the Location header. - FM
-                    */
-                   cp++;
-               }
-             }
-
-             /*
-              *  If we get to here, we didn't find the Location
-              *  header, so we'll show the user what we got, if
-              *  anything. - FM
-              */
-             CTRACE((tfp, "HTTP: Failed to pick up location.\n"));
-             doing_redirect = FALSE;
-             permanent_redirection = FALSE;
-             start_of_data = line_kept_clean;
-             length = rawlength;
-             if (!bad_location) {
-                 HTAlert(gettext("Got redirection with no Location header."));
-                 HTProgress(line_buffer);
-             }
-             if (traversal) {
-                 HTTP_NETCLOSE(s, handle);
-                 status = -1;
-                 goto clean_up;
-             }
-             if (!dump_output_immediately &&
-                 format_out == HTAtom_for("www/download")) {
-                 /*
-                  *  Convert a download request to
-                  *  a presentation request for
-                  *  interactive users. - FM
-                  */
-                 format_out = WWW_PRESENT;
-             }
              break;
           }
 
@@ -1901,6 +1538,13 @@
   */
   if (HTCheckForInterrupt()) {
       HTTP_NETCLOSE(s, handle);
+      if (doing_redirect) {
+         /*
+          *  Impatient user. - FM
+          */
+         CTRACE((tfp, "HTTP: Interrupted followup read.\n"));
+         _HTProgress (CONNECTION_INTERRUPTED);
+      }
       status = HT_INTERRUPTED;
       goto clean_up;
   }
@@ -1919,6 +1563,24 @@
       length = rawlength;
 #endif
       format_in = HTAtom_for("text/plain");
+
+  } else if (doing_redirect) {
+
+      format_in = HTAtom_for("message/x-http-redirection");
+      StrAllocCopy(anAnchor->content_type, HTAtom_name(format_in));
+      if (traversal) {
+         format_out = WWW_DEBUG;
+         if (!sink)
+             sink = HTErrorStream();
+      } else if (!dump_output_immediately &&
+         format_out == HTAtom_for("www/download")) {
+         /*
+          *  Convert a download request to
+          *  a presentation request for
+          *  interactive users. - FM
+          */
+         format_out = WWW_PRESENT;
+      }
   }
 
   target = HTStreamStack(format_in,
@@ -1946,12 +1608,23 @@
   */
   rv = HTCopy(anAnchor, s, (void *)handle, target);
 
+  /*
+  **  If we get here with doing_redirect set, it means that we were
+  **  looking for a Location header.  We either have got it now in
+  **  redirecting_url - in that case the stream should not have loaded
+  **  any data.  Or we didn't get it, in that case the stream may have
+  **  presented the message body normally. - kw
+  */
+
   if (rv == -1) {
       /*
       **  Intentional interrupt before data were received, not an error
       */
       /* (*target->isa->_abort)(target, NULL); */ /* already done in HTCopy */
-      status = HT_INTERRUPTED;
+      if (doing_redirect && traversal)
+         status = -1;
+      else
+         status = HT_INTERRUPTED;
       HTTP_NETCLOSE(s, handle);
       goto clean_up;
   }
@@ -1961,21 +1634,29 @@
       **  Aw hell, a REAL error, maybe cuz it's a dumb HTTP0 server
       */
       (*target->isa->_abort)(target, NULL);
-      HTTP_NETCLOSE(s, handle);
-      if (!already_retrying && !do_post) {
-         CTRACE((tfp, "HTTP: Trying again with HTTP0 request.\n"));
+      if (doing_redirect && redirecting_url) {
          /*
-         **  May as well consider it an interrupt -- right?
+         **  Got a location before the error occurred?  Then consider it
+         **  an interrupt but proceed below as normal. - kw
          */
-         FREE(line_buffer);
-         FREE(line_kept_clean);
-         extensions = NO;
-         already_retrying = TRUE;
-         _HTProgress (RETRYING_AS_HTTP0);
-         goto try_again;
+         /* do nothing here */
       } else {
-         status = HT_NOT_LOADED;
-         goto clean_up;
+         HTTP_NETCLOSE(s, handle);
+         if (!doing_redirect && !already_retrying && !do_post) {
+             CTRACE((tfp, "HTTP: Trying again with HTTP0 request.\n"));
+             /*
+             **  May as well consider it an interrupt -- right?
+             */
+             FREE(line_buffer);
+             FREE(line_kept_clean);
+             extensions = NO;
+             already_retrying = TRUE;
+             _HTProgress (RETRYING_AS_HTTP0);
+             goto try_again;
+         } else {
+             status = HT_NOT_LOADED;
+             goto clean_up;
+         }
       }
   }
 
@@ -1983,23 +1664,107 @@
   **  Free if complete transmission (socket was closed before return).
   **  Close socket if partial transmission (was freed on abort).
   */
-  if (rv != HT_INTERRUPTED) {
+  if (rv != HT_INTERRUPTED && rv != -2) {
       (*target->isa->_free)(target);
   } else {
       HTTP_NETCLOSE(s, handle);
   }
 
   if (doing_redirect) {
-      /*
-      **  We already jumped over all this if the "case 3:" code worked
-      **  above, but we'll check here as a backup in case it fails. - FM
-      */
-      /* Lou's old comment:  - FM */
-      /* OK, now we've got the redirection URL temporarily stored
-        in external variable redirecting_url, exported from HTMIME.c,
-        since there's no straightforward way to do this in the library
-        currently.  Do the right thing. */
-      status = HT_REDIRECTING;
+      if (redirecting_url) {
+         /*
+          *  Set up for checking redirecting_url in
+          *  LYGetFile.c for restrictions before we
+          *  seek the document at that Location. - FM
+          */
+         CTRACE((tfp, "HTTP: Picked up location '%s'\n",
+                 redirecting_url));
+         if (rv == HT_INTERRUPTED) {
+             /*        
+             **  Intentional interrupt after data were received, not an
+             **  error (probably).  We take it as a user request to
+             **  abandon the redirection chain.
+             **  This could reasonably be changed (by just removing this
+             **  block), it would make sense if there are redirecting
+             **  resources that "hang" after sending the headers. - kw
+             */
+             FREE(redirecting_url);
+             CTRACE((tfp, "HTTP: Interrupted followup read.\n"));
+             status = HT_INTERRUPTED;
+             goto clean_up;
+         }
+         HTProgress(line_buffer);
+         if (server_status == 305) { /* Use Proxy */
+             /*
+              *        Make sure the proxy field ends with
+              *        a slash. - FM
+              */
+             if (redirecting_url[strlen(redirecting_url)-1]
+                 != '/')
+                 StrAllocCat(redirecting_url, "/");
+             /*
+              *        Append our URL. - FM
+              */
+             StrAllocCat(redirecting_url, anAnchor->address);
+             CTRACE((tfp, "HTTP: Proxy URL is '%s'\n",
+                     redirecting_url));
+         }
+         if (!do_post ||
+             server_status == 303 ||
+             server_status == 302) {
+             /*
+              *        We don't have POST content (nor support PUT
+              *        or DELETE), or the status is "See Other"  or
+              *        "General Redirection" and we can convert to
+              *        GET, so go back and check out the new URL. - FM
+              */
+             status = HT_REDIRECTING;
+             goto clean_up;
+         }
+         /*
+          *  Make sure the user wants to redirect
+          *  the POST content, or treat as GET - FM & DK
+          */
+         switch (HTConfirmPostRedirect(redirecting_url,
+                                       server_status)) {
+             /*
+              *        User failed to confirm.
+              *        Abort the fetch.
+              */
+         case 0:
+             doing_redirect = FALSE;
+             FREE(redirecting_url);
+             status = HT_NO_DATA;
+             goto clean_up;
+
+             /*
+              *        User wants to treat as GET with no content.
+              *        Go back to check out the URL.
+              */
+         case 303:
+             break;
+
+             /*
+              *        Set the flag to retain the POST
+              *        content and go back to check out
+              *        the URL. - FM
+              */
+         default:
+             redirect_post_content = TRUE;
+         }
+
+         /* Lou's old comment:  - FM */
+         /* OK, now we've got the redirection URL temporarily stored
+            in external variable redirecting_url, exported from HTMIME.c,
+            since there's no straightforward way to do this in the library
+            currently.  Do the right thing. */
+
+         status = HT_REDIRECTING;
+
+      } else {
+         status = traversal ? -1 : HT_LOADED;
+      }
+
   } else {
       /*
       **  If any data were received, treat as a complete transmission
Index: 2.30/WWW/Library/Implementation/HTFormat.c
--- 2.30/WWW/Library/Implementation/HTFormat.c Sat, 04 Dec 1999 00:06:06 -0600
+++ 2.30(w)/WWW/Library/Implementation/HTFormat.c Sat, 04 Dec 1999 01:59:43 
-0600
@@ -598,6 +598,7 @@
        HTStream*,              sink)
 {
     HTStreamClass targetClass;
+    BOOL suppress_readprogress = NO;
     int bytes;
     int rv = 0;
 #ifdef _WINDOWS        /* 1997/11/11 (Tue) 15:18:16 */
@@ -715,6 +716,16 @@
            break;
        }
 
+       /*
+        *  Suppress ReadProgress messages when collecting a redirection
+        *  message, at least initially (unless/until anchor->content_type
+        *  gets changed, probably by the MIME message parser).  That way
+        *  messages put up by the HTTP module or elsewhere can linger in
+        *  the statusline for a while. - kw
+        */
+       suppress_readprogress = (anchor && anchor->content_type &&
+                                !strcmp(anchor->content_type,
+                                        "message/x-http-redirection"));
 #ifdef NOT_ASCII
        {
            char * p;
@@ -726,7 +737,8 @@
 
        (*targetClass.put_block)(sink, input_buffer, status);
        bytes += status;
-       HTReadProgress(bytes, anchor ? anchor->content_length : 0);
+       if (!suppress_readprogress)
+           HTReadProgress(bytes, anchor ? anchor->content_length : 0);
        HTDisplayPartial();
 
     } /* next bufferload */
@@ -795,7 +807,9 @@
        (*targetClass.put_block)(sink, input_buffer, status);
        bytes += status;
        HTReadProgress(bytes, 0);
-       HTDisplayPartial();
+       if (NumOfLines_partial >= 0 && HTMainText && HTMainAnchor &&
+           bytes != HTMainAnchor->content_length) 
+           HTDisplayPartial();
 
        if (HTCheckForInterrupt()) {
            _HTProgress (TRANSFER_INTERRUPTED);
@@ -1350,4 +1364,55 @@
     me->had_cr = NO;
     me->sink = sink;
     return me;
+}
+
+PRIVATE HTStream       HTBaseStreamInstance;                 /* Made static */
+/*
+**     ERROR STREAM
+**     ------------
+**     There is only one error stream shared by anyone who wants a
+**     generic error returned from all stream methods.
+*/
+PRIVATE void HTErrorStream_put_character ARGS2(HTStream *, me GCC_UNUSED, 
char, c GCC_UNUSED)
+{
+    LYCancelDownload = TRUE;
+}
+
+PRIVATE void HTErrorStream_put_string ARGS2(HTStream *, me GCC_UNUSED, CONST 
char *, s)
+{
+    if (s && *s)
+       LYCancelDownload = TRUE;
+}
+
+PRIVATE void HTErrorStream_write ARGS3(HTStream *, me GCC_UNUSED, CONST char 
*, s, int, l)
+{
+    if (l && s)
+       LYCancelDownload = TRUE;
+}
+
+PRIVATE void HTErrorStream_free ARGS1(HTStream *, me GCC_UNUSED)
+{
+    return;
+}
+
+PRIVATE void HTErrorStream_abort ARGS2(HTStream *, me GCC_UNUSED, HTError, e 
GCC_UNUSED)
+{
+    return;
+}
+
+PRIVATE CONST HTStreamClass HTErrorStreamClass =
+{              
+    "ErrorStream",
+    HTErrorStream_free,
+    HTErrorStream_abort,
+    HTErrorStream_put_character,
+    HTErrorStream_put_string,
+    HTErrorStream_write
+}; 
+
+PUBLIC HTStream * HTErrorStream (void)
+{
+    CTRACE((tfp, "ErrorStream. Created\n"));
+    HTBaseStreamInstance.isa = &HTErrorStreamClass;    /* The rest is random */
+    return &HTBaseStreamInstance;
 }
Index: 2.30/WWW/Library/Implementation/HTMIME.c
--- 2.30/WWW/Library/Implementation/HTMIME.c Wed, 06 Oct 1999 14:48:20 -0500
+++ 2.30(w)/WWW/Library/Implementation/HTMIME.c Sat, 04 Dec 1999 01:59:47 -0600
@@ -13,6 +13,7 @@
 */
 #include <HTUtils.h>
 #include <HTMIME.h>            /* Implemented here */
+#include <HTTP.h>              /* for redirecting_url */
 #include <HTAlert.h>
 #include <HTCJK.h>
 #include <UCMap.h>
@@ -114,6 +115,9 @@
        MIME_state              if_ok;          /* got this state if match */
        MIME_state              field;          /* remember which field */
        MIME_state              fold_state;     /* state on a fold */
+       BOOL                    head_only;      /* only parsing header */
+       BOOL                    pickup_redirection; /* parsing for location */
+       BOOL                    no_streamstack; /* use sink directly */
        CONST char *            check_pointer;  /* checking input */
 
        char *                  value_pointer;  /* storing values */
@@ -125,6 +129,7 @@
        char *                  boundary;       /* For multipart */
        char *                  set_cookie;     /* Set-Cookie */
        char *                  set_cookie2;    /* Set-Cookie2 */
+       char *                  location;       /* Location */
 
        HTFormat                encoding;       /* Content-Transfer-Encoding */
        char *                  compression_encoding;
@@ -161,6 +166,748 @@
        value[i] = cp[(i +1)];
 }
 
+PRIVATE int pumpData (HTStream * me)
+{
+    if (strchr(HTAtom_name(me->format), ';') != NULL) {
+       char *cp = NULL, *cp1, *cp2, *cp3 = NULL, *cp4;
+
+       CTRACE((tfp, "HTMIME: Extended MIME Content-Type is %s\n",
+               HTAtom_name(me->format)));
+       StrAllocCopy(cp, HTAtom_name(me->format));
+       /*
+       **      Note that the Content-Type value was converted
+       **      to lower case when we loaded into me->format,
+       **      but there may have been a mixed or upper-case
+       **      atom, so we'll force lower-casing again.  We
+       **      also stripped spaces and double-quotes, but
+       **      we'll make sure they're still gone from any
+       **      charset parameter we check. - FM
+       */
+       LYLowerCase(cp);
+       if ((cp1 = strchr(cp, ';')) != NULL) {
+           BOOL chartrans_ok = NO;
+           if ((cp2 = strstr(cp1, "charset")) != NULL) {
+               int chndl;
+
+               cp2 += 7;
+               while (*cp2 == ' ' || *cp2 == '=' || *cp2 == '\"')
+                   cp2++;
+               StrAllocCopy(cp3, cp2); /* copy to mutilate more */
+               for (cp4 = cp3; (*cp4 != '\0' && *cp4 != '\"' &&
+                                *cp4 != ';'  && *cp4 != ':' &&
+                                !WHITE(*cp4)); cp4++)
+                   ; /* do nothing */
+               *cp4 = '\0';
+               cp4 = cp3;
+               chndl = UCGetLYhndl_byMIME(cp3);
+               if (UCCanTranslateFromTo(chndl,
+                                        current_char_set)) {
+                   chartrans_ok = YES;
+                   *cp1 = '\0';
+                   me->format = HTAtom_for(cp);
+                   StrAllocCopy(me->anchor->charset, cp4);
+                   HTAnchor_setUCInfoStage(me->anchor, chndl,
+                                           UCT_STAGE_MIME,
+                                           UCT_SETBY_MIME);
+               }
+               else if (chndl < 0) {/* got something but we don't
+                                       recognize it */
+                   chndl = UCLYhndl_for_unrec;
+                   if (chndl < 0)
+                       /*
+                       **  UCLYhndl_for_unrec not defined :-(
+                       **  fallback to UCLYhndl_for_unspec
+                       **  which always valid.
+                       */
+                       chndl = UCLYhndl_for_unspec;  /* always >= 0 */
+                   if (UCCanTranslateFromTo(chndl,
+                                            current_char_set)) {
+                       chartrans_ok = YES;
+                       *cp1 = '\0';
+                       me->format = HTAtom_for(cp);
+                       HTAnchor_setUCInfoStage(me->anchor, chndl,
+                                               UCT_STAGE_MIME,
+                                               UCT_SETBY_DEFAULT);
+                   }
+               }
+               if (chartrans_ok) {
+                   LYUCcharset * p_in =
+                       HTAnchor_getUCInfoStage(me->anchor,
+                                               UCT_STAGE_MIME);
+                   LYUCcharset * p_out =
+                       HTAnchor_setUCInfoStage(me->anchor,
+                                               current_char_set,
+                                               UCT_STAGE_HTEXT,
+                                               UCT_SETBY_DEFAULT);
+                   if (!p_out)
+                       /*
+                       **      Try again.
+                       */
+                       p_out =
+                           HTAnchor_getUCInfoStage(me->anchor,
+                                                   UCT_STAGE_HTEXT);
+
+                   if (!strcmp(p_in->MIMEname,
+                               "x-transparent")) {
+                       HTPassEightBitRaw = TRUE;
+                       HTAnchor_setUCInfoStage(me->anchor,
+                                               HTAnchor_getUCLYhndl(me->anchor,
+                                                                    
UCT_STAGE_HTEXT),
+                                               UCT_STAGE_MIME,
+                                               UCT_SETBY_DEFAULT);
+                   }
+                   if (!strcmp(p_out->MIMEname,
+                               "x-transparent")) {
+                       HTPassEightBitRaw = TRUE;
+                       HTAnchor_setUCInfoStage(me->anchor,
+                                               HTAnchor_getUCLYhndl(me->anchor,
+                                                                    
UCT_STAGE_MIME),
+                                               UCT_STAGE_HTEXT,
+                                               UCT_SETBY_DEFAULT);
+                   }
+                   if (p_in->enc != UCT_ENC_CJK) {
+                       HTCJK = NOCJK;
+                       if (!(p_in->codepoints &
+                             UCT_CP_SUBSETOF_LAT1) &&
+                           chndl == current_char_set) {
+                           HTPassEightBitRaw = TRUE;
+                       }
+                   } else if (p_out->enc == UCT_ENC_CJK) {
+                       Set_HTCJK(p_in->MIMEname, p_out->MIMEname);
+                   }
+               } else {
+                   /*
+                   **  Cannot translate.
+                   **  If according to some heuristic the given
+                   **  charset and the current display character
+                   **  both are likely to be like ISO-8859 in
+                   **  structure, pretend we have some kind
+                   **  of match.
+                   */
+                   BOOL given_is_8859
+                       = (BOOL) (!strncmp(cp4, "iso-8859-", 9) &&
+                                 isdigit((unsigned char)cp4[9]));
+                   BOOL given_is_8859like
+                       = (BOOL) (given_is_8859 ||
+                                 !strncmp(cp4, "windows-", 8) ||
+                                 !strncmp(cp4, "cp12", 4) ||
+                                 !strncmp(cp4, "cp-12", 5));
+                   BOOL given_and_display_8859like
+                       = (BOOL) (given_is_8859like &&
+                                 (strstr(LYchar_set_names[current_char_set],
+                                         "ISO-8859") ||
+                                  strstr(LYchar_set_names[current_char_set],
+                                         "windows-")));
+
+                   if (given_and_display_8859like) {
+                       *cp1 = '\0';
+                       me->format = HTAtom_for(cp);
+                   }
+                   if (given_is_8859) {
+                       cp1 = &cp4[10];
+                       while (*cp1 &&
+                              isdigit((unsigned char)(*cp1)))
+                           cp1++;
+                       *cp1 = '\0';
+                   }
+                   if (given_and_display_8859like) {
+                       StrAllocCopy(me->anchor->charset, cp4);
+                       HTPassEightBitRaw = TRUE;
+                   }
+                   HTAlert(*cp4 ? cp4 : me->anchor->charset);
+               }
+               FREE(cp3);
+           } else {
+               /*
+               **      No charset parameter is present.
+               **      Ignore all other parameters, as
+               **      we do when charset is present. - FM
+               */
+               *cp1 = '\0';
+               me->format = HTAtom_for(cp);
+           }
+       }
+       FREE(cp);
+    }
+    /*
+    **  If we have an Expires header and haven't
+    **  already set the no_cache element for the
+    **  anchor, check if we should set it based
+    **  on that header. - FM
+    */
+    if (me->anchor->no_cache == FALSE &&
+       me->anchor->expires != NULL) {
+       if (!strcmp(me->anchor->expires, "0")) {
+           /*
+            *  The value is zero, which we treat as
+            *  an absolute no-cache directive. - FM
+            */
+           me->anchor->no_cache = TRUE;
+       } else if (me->anchor->date != NULL) {
+           /*
+           **  We have a Date header, so check if
+           **  the value is less than or equal to
+           **  that. - FM
+           */
+           if (LYmktime(me->anchor->expires, TRUE) <=
+               LYmktime(me->anchor->date, TRUE)) {
+               me->anchor->no_cache = TRUE;
+           }
+       } else if (LYmktime(me->anchor->expires, FALSE) <= 0) {
+           /*
+           **  We don't have a Date header, and
+           **  the value is in past for us. - FM
+           */
+           me->anchor->no_cache = TRUE;
+       }
+    }
+    StrAllocCopy(me->anchor->content_type,
+                HTAtom_name(me->format));
+
+    if (me->set_cookie != NULL || me->set_cookie2 != NULL) {
+       LYSetCookie(me->set_cookie,
+                   me->set_cookie2,
+                   me->anchor->address);
+       FREE(me->set_cookie);
+       FREE(me->set_cookie2);
+    }
+    if (me->pickup_redirection) {
+       if (me->location && *me->location) {
+           redirecting_url = me->location;
+           me->location = NULL;
+           if (me->targetRep != WWW_DEBUG || me->sink)
+               me->head_only = YES;
+
+       } else {
+           permanent_redirection = FALSE;
+           if (me->location) {
+               CTRACE((tfp, "HTTP: 'Location:' is zero-length!\n"));
+               HTAlert(
+                   gettext("Got redirection with a bad Location header."));
+           }
+           CTRACE((tfp, "HTTP: Failed to pick up location.\n"));
+           if (me->location) {
+               FREE(me->location);
+           } else {
+               HTAlert(gettext("Got redirection with no Location header."));
+           }
+       }
+    }
+    if (me->head_only) {
+       /* We are done! - kw */
+       me->state = MIME_IGNORE;
+       return HT_OK;
+    }
+
+    if (me->no_streamstack) {
+       me->target = me->sink;
+    } else {
+       if (!me->compression_encoding) {
+           CTRACE((tfp, "HTMIME: MIME Content-Type is '%s', converting to 
'%s'\n",
+                   HTAtom_name(me->format), HTAtom_name(me->targetRep)));
+       } else {
+           /*
+           **  Change the format to that for "www/compressed"
+           **  and set up a stream to deal with it. - FM
+           */
+           CTRACE((tfp, "HTMIME: MIME Content-Type is '%s',\n", 
HTAtom_name(me->format)));
+           me->format = HTAtom_for("www/compressed");
+           CTRACE((tfp, "        Treating as '%s'.  Converting to '%s'\n",
+                   HTAtom_name(me->format), HTAtom_name(me->targetRep)));
+           FREE(me->compression_encoding);
+       }
+       me->target = HTStreamStack(me->format, me->targetRep,
+                                  me->sink , me->anchor);
+       if (!me->target) {
+           CTRACE((tfp, "HTMIME: Can't translate! ** \n"));
+           me->target = me->sink;      /* Cheat */
+       }
+    }
+    if (me->target) {
+       me->targetClass = *me->target->isa;
+       /*
+       **      Check for encoding and select state from there,
+       **      someday, but until we have the relevant code,
+       **      from now push straight through. - FM
+       */
+       me->state = MIME_TRANSPARENT;   /* Pump rest of data right through */
+    } else {
+       me->state = MIME_IGNORE;        /* What else to do? */
+    }
+    return HT_OK;
+}
+
+PRIVATE int dispatchField (HTStream * me)
+{
+    int i, j;
+    char *cp;
+
+    *me->value_pointer = '\0';
+    cp = me->value_pointer;
+    while ((cp > me->value) && *(--cp) == ' ')  /* S/390 -- gil -- 0146 */
+       /*
+       **  Trim trailing spaces.
+       */
+       *cp = '\0';
+
+    switch (me->field) {
+    case miACCEPT_RANGES:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Accept-Ranges: '%s'\n",
+               me->value));
+       break;
+    case miAGE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Age: '%s'\n",
+               me->value));
+       break;
+    case miALLOW:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Allow: '%s'\n",
+               me->value));
+       break;
+    case miALTERNATES:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Alternates: '%s'\n",
+               me->value));
+       break;
+    case miCACHE_CONTROL:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Cache-Control: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Convert to lowercase and indicate in anchor. - FM
+       */
+       LYLowerCase(me->value);
+       StrAllocCopy(me->anchor->cache_control, me->value);
+       /*
+       **  Check whether to set no_cache for the anchor. - FM
+       */
+       {
+           char *cp1, *cp0 = me->value;
+
+           while ((cp1 = strstr(cp0, "no-cache")) != NULL) {
+               cp1 += 8;
+               while (*cp1 != '\0' && WHITE(*cp1))
+                   cp1++;
+               if (*cp1 == '\0' || *cp1 == ';') {
+                   me->anchor->no_cache = TRUE;
+                   break;
+               }
+               cp0 = cp1;
+           }
+           if (me->anchor->no_cache == TRUE)
+               break;
+           cp0 = me->value;
+           while ((cp1 = strstr(cp0, "max-age")) != NULL) {
+               cp1 += 7;
+               while (*cp1 != '\0' && WHITE(*cp1))
+                   cp1++;
+               if (*cp1 == '=') {
+                   cp1++;
+                   while (*cp1 != '\0' && WHITE(*cp1))
+                       cp1++;
+                   if (isdigit((unsigned char)*cp1)) {
+                       cp0 = cp1;
+                       while (isdigit((unsigned char)*cp1))
+                           cp1++;
+                       if (*cp0 == '0' && cp1 == (cp0 + 1)) {
+                           me->anchor->no_cache = TRUE;
+                           break;
+                       }
+                   }
+               }
+               cp0 = cp1;
+           }
+       }
+       break;
+    case miCOOKIE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Cookie: '%s'\n",
+               me->value));
+       break;
+    case miCONNECTION:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Connection: '%s'\n",
+               me->value));
+       break;
+    case miCONTENT_BASE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Base: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->content_base, me->value);
+       break;
+    case miCONTENT_DISPOSITION:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Disposition: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->content_disposition, me->value);
+       /*
+       **  It's not clear yet from existing RFCs and IDs
+       **  whether we should be looking for file;, attachment;,
+       **  and/or inline; before the filename=value, so we'll
+       **  just search for "filename" followed by '=' and just
+       **  hope we get the intended value.  It is purely a
+       **  suggested name, anyway. - FM
+       */
+       cp = me->anchor->content_disposition;
+       while (*cp != '\0' && strncasecomp(cp, "filename", 8))
+           cp++;
+       if (*cp == '\0')
+           break;
+       cp += 8;
+       while ((*cp != '\0') && (WHITE(*cp) || *cp == '='))
+           cp++;
+       if (*cp == '\0')
+           break;
+       while (*cp != '\0' && WHITE(*cp))
+           cp++;
+       if (*cp == '\0')
+           break;
+       StrAllocCopy(me->anchor->SugFname, cp);
+       if (*me->anchor->SugFname == '\"') {
+           if ((cp = strchr((me->anchor->SugFname + 1),
+                            '\"')) != NULL) {
+               *(cp + 1) = '\0';
+               HTMIME_TrimDoubleQuotes(me->anchor->SugFname);
+           } else {
+               FREE(me->anchor->SugFname);
+               break;
+           }
+       }
+       cp = me->anchor->SugFname;
+       while (*cp != '\0' && !WHITE(*cp))
+           cp++;
+       *cp = '\0';
+       if (*me->anchor->SugFname == '\0')
+           FREE(me->anchor->SugFname);
+       break;
+    case miCONTENT_ENCODING:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Encoding: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value) ||
+           !strcasecomp(me->value, "identity"))
+           break;
+       /*
+       **  Convert to lowercase and indicate in anchor. - FM
+       */
+       LYLowerCase(me->value);
+       StrAllocCopy(me->anchor->content_encoding, me->value);
+       FREE(me->compression_encoding);
+       if (!strcmp(me->value, "8bit") ||
+           !strcmp(me->value, "7bit") ||
+           !strcmp(me->value, "binary")) {
+           /*
+           **  Some server indicated "8bit", "7bit" or "binary"
+           **  inappropriately.  We'll ignore it. - FM
+           */
+           CTRACE((tfp, "                Ignoring it!\n"));
+       } else {
+           /*
+           **  Save it to use as a flag for setting
+           **  up a "www/compressed" target. - FM
+           */
+           StrAllocCopy(me->compression_encoding, me->value);
+       }
+       break;
+    case miCONTENT_FEATURES:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Features: '%s'\n",
+               me->value));
+       break;
+    case miCONTENT_LANGUAGE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Language: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Convert to lowercase and indicate in anchor. - FM
+       */
+       LYLowerCase(me->value);
+       StrAllocCopy(me->anchor->content_language, me->value);
+       break;
+    case miCONTENT_LENGTH:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Length: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Convert to integer and indicate in anchor. - FM
+       */
+       me->anchor->content_length = atoi(me->value);
+       if (me->anchor->content_length < 0)
+           me->anchor->content_length = 0;
+       CTRACE((tfp, "        Converted to integer: '%d'\n",
+               me->anchor->content_length));
+       break;
+    case miCONTENT_LOCATION:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Location: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->content_location, me->value);
+       break;
+    case miCONTENT_MD5:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-MD5: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->content_md5, me->value);
+       break;
+    case miCONTENT_RANGE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Range: '%s'\n",
+               me->value));
+       break;
+    case miCONTENT_TRANSFER_ENCODING:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Transfer-Encoding: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Force the Content-Transfer-Encoding value
+       **  to all lower case. - FM
+       */
+       LYLowerCase(me->value);
+       me->encoding = HTAtom_for(me->value);
+       break;
+    case miCONTENT_TYPE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Content-Type: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Force the Content-Type value to all lower case
+       **  and strip spaces and double-quotes. - FM
+       */
+       for (i = 0, j = 0; me->value[i]; i++) {
+           if (me->value[i] != ' ' && me->value[i] != '\"') {
+               me->value[j++] = (char) TOLOWER(me->value[i]);
+           }
+       }
+       me->value[j] = '\0';
+       me->format = HTAtom_for(me->value);
+       break;
+    case miDATE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Date: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->date, me->value);
+       break;
+    case miETAG:
+       /*  Do not trim double quotes:
+        *  an entity tag consists of an opaque quoted string,
+        *  possibly prefixed by a weakness indicator.
+        */
+       CTRACE((tfp, "HTMIME: PICKED UP ETag: %s\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->ETag, me->value);
+       break;
+    case miEXPIRES:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Expires: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->expires, me->value);
+       break;
+    case miKEEP_ALIVE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Keep-Alive: '%s'\n",
+               me->value));
+       break;
+    case miLAST_MODIFIED:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Last-Modified: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->last_modified, me->value);
+       break;
+    case miLINK:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Link: '%s'\n",
+               me->value));
+       break;
+    case miLOCATION:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Location: '%s'\n",
+               me->value));
+       if (me->pickup_redirection && !me->location) {
+           StrAllocCopy(me->location, me->value);
+       } else {
+           CTRACE((tfp, "HTMIME: *** Ignoring Location!\n"));
+       }
+       break;
+    case miPRAGMA:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Pragma: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Check whether to set no_cache for the anchor. - FM
+       */
+       if (!strcmp(me->value, "no-cache"))
+           me->anchor->no_cache = TRUE;
+       break;
+    case miPROXY_AUTHENTICATE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Proxy-Authenticate: '%s'\n",
+               me->value));
+       break;
+    case miPUBLIC:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Public: '%s'\n",
+               me->value));
+       break;
+    case miRETRY_AFTER:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Retry-After: '%s'\n",
+               me->value));
+       break;
+    case miSAFE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Safe: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor if "YES" or "TRUE". - FM
+       */
+       if (!strcasecomp(me->value, "YES") ||
+           !strcasecomp(me->value, "TRUE")) {
+           me->anchor->safe = TRUE;
+       } else if (!strcasecomp(me->value, "NO") ||
+                  !strcasecomp(me->value, "FALSE")) {
+           /*
+           **  If server explicitly tells us that it has changed
+           **  its mind, reset flag in anchor. - kw
+           */
+           me->anchor->safe = FALSE;
+       }
+       break;
+    case miSERVER:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Server: '%s'\n",
+               me->value));
+       if (!(me->value && *me->value))
+           break;
+       /*
+       **  Indicate in anchor. - FM
+       */
+       StrAllocCopy(me->anchor->server, me->value);
+       break;
+    case miSET_COOKIE1:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Set-Cookie: '%s'\n",
+               me->value));
+       if (me->set_cookie == NULL) {
+           StrAllocCopy(me->set_cookie, me->value);
+       } else {
+           StrAllocCat(me->set_cookie, ", ");
+           StrAllocCat(me->set_cookie, me->value);
+       }
+       break;
+    case miSET_COOKIE2:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Set-Cookie2: '%s'\n",
+               me->value));
+       if (me->set_cookie2 == NULL) {
+           StrAllocCopy(me->set_cookie2, me->value);
+       } else {
+           StrAllocCat(me->set_cookie2, ", ");
+           StrAllocCat(me->set_cookie2, me->value);
+       }
+       break;
+    case miTITLE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Title: '%s'\n",
+               me->value));
+       break;
+    case miTRANSFER_ENCODING:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Transfer-Encoding: '%s'\n",
+               me->value));
+       break;
+    case miUPGRADE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Upgrade: '%s'\n",
+               me->value));
+       break;
+    case miURI:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP URI: '%s'\n",
+               me->value));
+       break;
+    case miVARY:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Vary: '%s'\n",
+               me->value));
+       break;
+    case miVIA:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Via: '%s'\n",
+               me->value));
+       break;
+    case miWARNING:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP Warning: '%s'\n",
+               me->value));
+       break;
+    case miWWW_AUTHENTICATE:
+       HTMIME_TrimDoubleQuotes(me->value);
+       CTRACE((tfp, "HTMIME: PICKED UP WWW-Authenticate: '%s'\n",
+               me->value));
+       break;
+    default:           /* Should never get here */
+       return HT_ERROR;
+    }
+    return HT_OK;
+}
+
+
 /*_________________________________________________________________________
 **
 **                     A C T I O N     R O U T I N E S
@@ -169,16 +916,15 @@
 /*     Character handling
 **     ------------------
 **
-**     This is a FSM parser which is tolerant as it can be of all
-**     syntax errors.  It ignores field names it does not understand,
-**     and resynchronises on line beginnings.
+**     This is a FSM parser. It ignores field names it does not understand.
+**     Folded header fields are recognized.  Lines without a fieldname at
+**     the beginning (that are not folded continuation lines) are ignored
+**     as unknown field names.  Fields with empty values are not picked up.
 */
 PRIVATE void HTMIME_put_character ARGS2(
        HTStream *,     me,
        char,           c)
 {
-    int i, j;
-
     if (me->state == MIME_TRANSPARENT) {
        (*me->targetClass.put_character)(me->target, c);/* MUST BE FAST */
        return;
@@ -223,7 +969,17 @@
     case miNEWLINE:
        if (c != '\n' && WHITE(c)) {            /* Folded line */
            me->state = me->fold_state; /* pop state before newline */
+           if (me->state == miGET_VALUE &&
+               me->value_pointer && me->value_pointer != me->value &&
+               !WHITE(*(me->value_pointer-1))) {
+               c = ' ';
+               goto GET_VALUE; /* will add space to value if it fits - kw */
+           }
            break;
+       } else if (me->fold_state == miGET_VALUE) {
+           /* Got a field, and now we know it's complete - so
+            * act on it. - kw */
+           dispatchField(me);
        }
 
        /*      else Falls through */
@@ -318,239 +1074,7 @@
        case '\n':                      /* Blank line: End of Header! */
            {
                me->net_ascii = NO;
-               if (strchr(HTAtom_name(me->format), ';') != NULL) {
-                   char *cp = NULL, *cp1, *cp2, *cp3 = NULL, *cp4;
-
-                   CTRACE((tfp, "HTMIME: Extended MIME Content-Type is %s\n",
-                               HTAtom_name(me->format)));
-                   StrAllocCopy(cp, HTAtom_name(me->format));
-                   /*
-                   **  Note that the Content-Type value was converted
-                   **  to lower case when we loaded into me->format,
-                   **  but there may have been a mixed or upper-case
-                   **  atom, so we'll force lower-casing again.  We
-                   **  also stripped spaces and double-quotes, but
-                   **  we'll make sure they're still gone from any
-                   **  charset parameter we check. - FM
-                   */
-                   LYLowerCase(cp);
-                   if ((cp1 = strchr(cp, ';')) != NULL) {
-                       BOOL chartrans_ok = NO;
-                       if ((cp2 = strstr(cp1, "charset")) != NULL) {
-                           int chndl;
-
-                           cp2 += 7;
-                           while (*cp2 == ' ' || *cp2 == '=' || *cp2 == '\"')
-                               cp2++;
-                           StrAllocCopy(cp3, cp2); /* copy to mutilate more */
-                           for (cp4 = cp3; (*cp4 != '\0' && *cp4 != '\"' &&
-                                            *cp4 != ';'  && *cp4 != ':' &&
-                                            !WHITE(*cp4));     cp4++)
-                               ; /* do nothing */
-                           *cp4 = '\0';
-                           cp4 = cp3;
-                           chndl = UCGetLYhndl_byMIME(cp3);
-                           if (UCCanTranslateFromTo(chndl,
-                                                    current_char_set)) {
-                               chartrans_ok = YES;
-                               *cp1 = '\0';
-                               me->format = HTAtom_for(cp);
-                               StrAllocCopy(me->anchor->charset, cp4);
-                               HTAnchor_setUCInfoStage(me->anchor, chndl,
-                                                       UCT_STAGE_MIME,
-                                                       UCT_SETBY_MIME);
-                           }
-                           else if (chndl < 0) {/* got something but we don't
-                                                recognize it */
-                               chndl = UCLYhndl_for_unrec;
-                               if (chndl < 0)
-                               /*
-                                **  UCLYhndl_for_unrec not defined :-(
-                                **  fallback to UCLYhndl_for_unspec
-                                **  which always valid.
-                                */
-                               chndl = UCLYhndl_for_unspec;  /* always >= 0 */
-                               if (UCCanTranslateFromTo(chndl,
-                                                        current_char_set)) {
-                                   chartrans_ok = YES;
-                                   *cp1 = '\0';
-                                   me->format = HTAtom_for(cp);
-                                   HTAnchor_setUCInfoStage(me->anchor, chndl,
-                                                           UCT_STAGE_MIME,
-                                                           UCT_SETBY_DEFAULT);
-                               }
-                           }
-                           if (chartrans_ok) {
-                               LYUCcharset * p_in =
-                                   HTAnchor_getUCInfoStage(me->anchor,
-                                                           UCT_STAGE_MIME);
-                               LYUCcharset * p_out =
-                                   HTAnchor_setUCInfoStage(me->anchor,
-                                                           current_char_set,
-                                                           UCT_STAGE_HTEXT,
-                                                           UCT_SETBY_DEFAULT);
-                               if (!p_out)
-                                   /*
-                                   **  Try again.
-                                   */
-                                   p_out =
-                                     HTAnchor_getUCInfoStage(me->anchor,
-                                                             UCT_STAGE_HTEXT);
-
-                               if (!strcmp(p_in->MIMEname,
-                                           "x-transparent")) {
-                                   HTPassEightBitRaw = TRUE;
-                                   HTAnchor_setUCInfoStage(me->anchor,
-                                      HTAnchor_getUCLYhndl(me->anchor,
-                                                           UCT_STAGE_HTEXT),
-                                                           UCT_STAGE_MIME,
-                                                           UCT_SETBY_DEFAULT);
-                               }
-                               if (!strcmp(p_out->MIMEname,
-                                           "x-transparent")) {
-                                   HTPassEightBitRaw = TRUE;
-                                   HTAnchor_setUCInfoStage(me->anchor,
-                                        HTAnchor_getUCLYhndl(me->anchor,
-                                                             UCT_STAGE_MIME),
-                                                           UCT_STAGE_HTEXT,
-                                                           UCT_SETBY_DEFAULT);
-                               }
-                               if (p_in->enc != UCT_ENC_CJK) {
-                                   HTCJK = NOCJK;
-                                   if (!(p_in->codepoints &
-                                         UCT_CP_SUBSETOF_LAT1) &&
-                                       chndl == current_char_set) {
-                                       HTPassEightBitRaw = TRUE;
-                                   }
-                               } else if (p_out->enc == UCT_ENC_CJK) {
-                                   Set_HTCJK(p_in->MIMEname, p_out->MIMEname);
-                               }
-                           } else {
-                               /*
-                               **  Cannot translate.
-                               **  If according to some heuristic the given
-                               **  charset and the current display character
-                               **  both are likely to be like ISO-8859 in
-                               **  structure, pretend we have some kind
-                               **  of match.
-                               */
-                               BOOL given_is_8859
-                                   = (BOOL) (!strncmp(cp4, "iso-8859-", 9) &&
-                                      isdigit((unsigned char)cp4[9]));
-                               BOOL given_is_8859like
-                                   = (BOOL) (given_is_8859 ||
-                                      !strncmp(cp4, "windows-", 8) ||
-                                      !strncmp(cp4, "cp12", 4) ||
-                                      !strncmp(cp4, "cp-12", 5));
-                               BOOL given_and_display_8859like
-                                   = (BOOL) (given_is_8859like &&
-                                      
(strstr(LYchar_set_names[current_char_set],
-                                              "ISO-8859") ||
-                                       
strstr(LYchar_set_names[current_char_set],
-                                              "windows-")));
-
-                               if (given_and_display_8859like) {
-                                   *cp1 = '\0';
-                                   me->format = HTAtom_for(cp);
-                               }
-                               if (given_is_8859) {
-                                   cp1 = &cp4[10];
-                                   while (*cp1 &&
-                                          isdigit((unsigned char)(*cp1)))
-                                       cp1++;
-                                   *cp1 = '\0';
-                               }
-                               if (given_and_display_8859like) {
-                                   StrAllocCopy(me->anchor->charset, cp4);
-                                   HTPassEightBitRaw = TRUE;
-                               }
-                               HTAlert(*cp4 ? cp4 : me->anchor->charset);
-                           }
-                           FREE(cp3);
-                       } else {
-                           /*
-                           **  No charset parameter is present.
-                           **  Ignore all other parameters, as
-                           **  we do when charset is present. - FM
-                           */
-                           *cp1 = '\0';
-                           me->format = HTAtom_for(cp);
-                       }
-                   }
-                   FREE(cp);
-               }
-               /*
-               **  If we have an Expires header and haven't
-               **  already set the no_cache element for the
-               **  anchor, check if we should set it based
-               **  on that header. - FM
-               */
-               if (me->anchor->no_cache == FALSE &&
-                   me->anchor->expires != NULL) {
-                   if (!strcmp(me->anchor->expires, "0")) {
-                       /*
-                        *  The value is zero, which we treat as
-                        *  an absolute no-cache directive. - FM
-                        */
-                       me->anchor->no_cache = TRUE;
-                   } else if (me->anchor->date != NULL) {
-                       /*
-                       **  We have a Date header, so check if
-                       **  the value is less than or equal to
-                       **  that. - FM
-                       */
-                       if (LYmktime(me->anchor->expires, TRUE) <=
-                           LYmktime(me->anchor->date, TRUE)) {
-                           me->anchor->no_cache = TRUE;
-                       }
-                   } else if (LYmktime(me->anchor->expires, FALSE) <= 0) {
-                       /*
-                       **  We don't have a Date header, and
-                       **  the value is in past for us. - FM
-                       */
-                       me->anchor->no_cache = TRUE;
-                   }
-               }
-               StrAllocCopy(me->anchor->content_type,
-                            HTAtom_name(me->format));
-               if (!me->compression_encoding) {
-                   CTRACE((tfp, "HTMIME: MIME Content-Type is '%s', converting 
to '%s'\n",
-                               HTAtom_name(me->format), 
HTAtom_name(me->targetRep)));
-               } else {
-                   /*
-                   **  Change the format to that for "www/compressed"
-                   **  and set up a stream to deal with it. - FM
-                   */
-                   CTRACE((tfp, "HTMIME: MIME Content-Type is '%s',\n", 
HTAtom_name(me->format)));
-                   me->format = HTAtom_for("www/compressed");
-                   CTRACE((tfp, "        Treating as '%s'.  Converting to 
'%s'\n",
-                               HTAtom_name(me->format), 
HTAtom_name(me->targetRep)));
-               }
-               if (me->set_cookie != NULL || me->set_cookie2 != NULL) {
-                   LYSetCookie(me->set_cookie,
-                               me->set_cookie2,
-                               me->anchor->address);
-                   FREE(me->set_cookie);
-                   FREE(me->set_cookie2);
-               }
-               me->target = HTStreamStack(me->format, me->targetRep,
-                                          me->sink , me->anchor);
-               if (!me->target) {
-                   CTRACE((tfp, "HTMIME: Can't translate! ** \n"));
-                   me->target = me->sink;      /* Cheat */
-               }
-               if (me->target) {
-                   me->targetClass = *me->target->isa;
-                   /*
-                   **  Check for encoding and select state from there,
-                   **  someday, but until we have the relevant code,
-                   **  from now push straight through. - FM
-                   */
-                   me->state = MIME_TRANSPARENT;
-               } else {
-                   me->state = MIME_IGNORE;    /* What else to do? */
-               }
-               FREE(me->compression_encoding);
+               pumpData(me);
            }
            break;
 
@@ -1170,9 +1694,9 @@
 
     case miSKIP_GET_VALUE:
        if (c == '\n') {
-          me->fold_state = me->state;
-          me->state = miNEWLINE;
-          break;
+           me->fold_state = me->state;
+           me->state = miNEWLINE;
+           break;
        }
        if (WHITE(c))
            /*
@@ -1185,465 +1709,8 @@
        /* Fall through to store first character */
 
     case miGET_VALUE:
-       if (WHITE(c) && c != ' ') {                     /* End of field */
-           char *cp;
-           *me->value_pointer = '\0';
-           cp = (me->value_pointer - 1);
-           while ((cp >= me->value) && *cp == ' ')  /* S/390 -- gil -- 0146 */
-               /*
-               **  Trim trailing spaces.
-               */
-               *cp = '\0';
-           switch (me->field) {
-           case miACCEPT_RANGES:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Accept-Ranges: '%s'\n",
-                           me->value));
-               break;
-           case miAGE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Age: '%s'\n",
-                           me->value));
-               break;
-           case miALLOW:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Allow: '%s'\n",
-                           me->value));
-               break;
-           case miALTERNATES:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Alternates: '%s'\n",
-                           me->value));
-               break;
-           case miCACHE_CONTROL:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Cache-Control: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Convert to lowercase and indicate in anchor. - FM
-               */
-               LYLowerCase(me->value);
-               StrAllocCopy(me->anchor->cache_control, me->value);
-               /*
-               **  Check whether to set no_cache for the anchor. - FM
-               */
-               {
-                   char *cp1, *cp0 = me->value;
-
-                   while ((cp1 = strstr(cp0, "no-cache")) != NULL) {
-                       cp1 += 8;
-                       while (*cp1 != '\0' && WHITE(*cp1))
-                           cp1++;
-                       if (*cp1 == '\0' || *cp1 == ';') {
-                           me->anchor->no_cache = TRUE;
-                           break;
-                       }
-                       cp0 = cp1;
-                   }
-                   if (me->anchor->no_cache == TRUE)
-                       break;
-                   cp0 = me->value;
-                   while ((cp1 = strstr(cp0, "max-age")) != NULL) {
-                       cp1 += 7;
-                       while (*cp1 != '\0' && WHITE(*cp1))
-                           cp1++;
-                       if (*cp1 == '=') {
-                           cp1++;
-                           while (*cp1 != '\0' && WHITE(*cp1))
-                               cp1++;
-                           if (isdigit((unsigned char)*cp1)) {
-                               cp0 = cp1;
-                               while (isdigit((unsigned char)*cp1))
-                                   cp1++;
-                               if (*cp0 == '0' && cp1 == (cp0 + 1)) {
-                                   me->anchor->no_cache = TRUE;
-                                   break;
-                               }
-                           }
-                       }
-                       cp0 = cp1;
-                   }
-               }
-               break;
-           case miCOOKIE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Cookie: '%s'\n",
-                           me->value));
-               break;
-           case miCONNECTION:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Connection: '%s'\n",
-                           me->value));
-               break;
-           case miCONTENT_BASE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Base: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->content_base, me->value);
-               break;
-           case miCONTENT_DISPOSITION:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Disposition: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->content_disposition, me->value);
-               /*
-               **  It's not clear yet from existing RFCs and IDs
-               **  whether we should be looking for file;, attachment;,
-               **  and/or inline; before the filename=value, so we'll
-               **  just search for "filename" followed by '=' and just
-               **  hope we get the intended value.  It is purely a
-               **  suggested name, anyway. - FM
-               */
-               cp = me->anchor->content_disposition;
-               while (*cp != '\0' && strncasecomp(cp, "filename", 8))
-                   cp++;
-               if (*cp == '\0')
-                   break;
-               cp += 8;
-               while ((*cp != '\0') && (WHITE(*cp) || *cp == '='))
-                   cp++;
-               if (*cp == '\0')
-                   break;
-               while (*cp != '\0' && WHITE(*cp))
-                   cp++;
-               if (*cp == '\0')
-                   break;
-               StrAllocCopy(me->anchor->SugFname, cp);
-               if (*me->anchor->SugFname == '\"') {
-                   if ((cp = strchr((me->anchor->SugFname + 1),
-                                    '\"')) != NULL) {
-                       *(cp + 1) = '\0';
-                       HTMIME_TrimDoubleQuotes(me->anchor->SugFname);
-                   } else {
-                       FREE(me->anchor->SugFname);
-                       break;
-                   }
-               }
-               cp = me->anchor->SugFname;
-               while (*cp != '\0' && !WHITE(*cp))
-                   cp++;
-               *cp = '\0';
-               if (*me->anchor->SugFname == '\0')
-                   FREE(me->anchor->SugFname);
-               break;
-           case miCONTENT_ENCODING:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Encoding: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value) ||
-                   !strcasecomp(me->value, "identity"))
-                   break;
-               /*
-               **  Convert to lowercase and indicate in anchor. - FM
-               */
-               LYLowerCase(me->value);
-               StrAllocCopy(me->anchor->content_encoding, me->value);
-               FREE(me->compression_encoding);
-               if (!strcmp(me->value, "8bit") ||
-                   !strcmp(me->value, "7bit") ||
-                   !strcmp(me->value, "binary")) {
-                   /*
-                   **  Some server indicated "8bit", "7bit" or "binary"
-                   **  inappropriately.  We'll ignore it. - FM
-                   */
-                   CTRACE((tfp, "                Ignoring it!\n"));
-               } else {
-                   /*
-                   **  Save it to use as a flag for setting
-                   **  up a "www/compressed" target. - FM
-                   */
-                   StrAllocCopy(me->compression_encoding, me->value);
-               }
-               break;
-           case miCONTENT_FEATURES:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Features: '%s'\n",
-                           me->value));
-               break;
-           case miCONTENT_LANGUAGE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Language: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Convert to lowercase and indicate in anchor. - FM
-               */
-               LYLowerCase(me->value);
-               StrAllocCopy(me->anchor->content_language, me->value);
-               break;
-           case miCONTENT_LENGTH:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Length: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Convert to integer and indicate in anchor. - FM
-               */
-               me->anchor->content_length = atoi(me->value);
-               if (me->anchor->content_length < 0)
-                   me->anchor->content_length = 0;
-               CTRACE((tfp, "        Converted to integer: '%d'\n",
-                           me->anchor->content_length));
-               break;
-           case miCONTENT_LOCATION:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Location: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->content_location, me->value);
-               break;
-           case miCONTENT_MD5:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-MD5: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->content_md5, me->value);
-               break;
-           case miCONTENT_RANGE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Range: '%s'\n",
-                           me->value));
-               break;
-           case miCONTENT_TRANSFER_ENCODING:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Transfer-Encoding: 
'%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Force the Content-Transfer-Encoding value
-               **  to all lower case. - FM
-               */
-               LYLowerCase(me->value);
-               me->encoding = HTAtom_for(me->value);
-               break;
-           case miCONTENT_TYPE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Content-Type: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Force the Content-Type value to all lower case
-               **  and strip spaces and double-quotes. - FM
-               */
-               for (i = 0, j = 0; me->value[i]; i++) {
-                   if (me->value[i] != ' ' && me->value[i] != '\"') {
-                       me->value[j++] = (char) TOLOWER(me->value[i]);
-                   }
-               }
-               me->value[j] = '\0';
-               me->format = HTAtom_for(me->value);
-               break;
-           case miDATE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Date: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->date, me->value);
-               break;
-           case miETAG:
-               /*  Do not trim double quotes:
-                *  an entity tag consists of an opaque quoted string,
-                *  possibly prefixed by a weakness indicator.
-                */
-               CTRACE((tfp, "HTMIME: PICKED UP ETag: %s\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->ETag, me->value);
-               break;
-           case miEXPIRES:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Expires: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->expires, me->value);
-               break;
-           case miKEEP_ALIVE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Keep-Alive: '%s'\n",
-                           me->value));
-               break;
-           case miLAST_MODIFIED:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Last-Modified: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->last_modified, me->value);
-               break;
-           case miLINK:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Link: '%s'\n",
-                           me->value));
-               break;
-           case miLOCATION:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Location: '%s'\n",
-                           me->value));
-               break;
-           case miPRAGMA:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Pragma: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Check whether to set no_cache for the anchor. - FM
-               */
-               if (!strcmp(me->value, "no-cache"))
-                   me->anchor->no_cache = TRUE;
-               break;
-           case miPROXY_AUTHENTICATE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Proxy-Authenticate: '%s'\n",
-                           me->value));
-               break;
-           case miPUBLIC:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Public: '%s'\n",
-                           me->value));
-               break;
-           case miRETRY_AFTER:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Retry-After: '%s'\n",
-                           me->value));
-               break;
-           case miSAFE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Safe: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor if "YES" or "TRUE". - FM
-               */
-               if (!strcasecomp(me->value, "YES") ||
-                   !strcasecomp(me->value, "TRUE")) {
-                   me->anchor->safe = TRUE;
-               } else if (!strcasecomp(me->value, "NO") ||
-                   !strcasecomp(me->value, "FALSE")) {
-                   /*
-                   **  If server explicitly tells us that it has changed
-                   **  its mind, reset flag in anchor. - kw
-                   */
-                   me->anchor->safe = FALSE;
-               }
-               break;
-           case miSERVER:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Server: '%s'\n",
-                           me->value));
-               if (!(me->value && *me->value))
-                   break;
-               /*
-               **  Indicate in anchor. - FM
-               */
-               StrAllocCopy(me->anchor->server, me->value);
-               break;
-           case miSET_COOKIE1:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Set-Cookie: '%s'\n",
-                           me->value));
-               if (me->set_cookie == NULL) {
-                   StrAllocCopy(me->set_cookie, me->value);
-               } else {
-                   StrAllocCat(me->set_cookie, ", ");
-                   StrAllocCat(me->set_cookie, me->value);
-               }
-               break;
-           case miSET_COOKIE2:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Set-Cookie2: '%s'\n",
-                           me->value));
-               if (me->set_cookie2 == NULL) {
-                   StrAllocCopy(me->set_cookie2, me->value);
-               } else {
-                   StrAllocCat(me->set_cookie2, ", ");
-                   StrAllocCat(me->set_cookie2, me->value);
-               }
-               break;
-           case miTITLE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Title: '%s'\n",
-                           me->value));
-               break;
-           case miTRANSFER_ENCODING:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Transfer-Encoding: '%s'\n",
-                           me->value));
-               break;
-           case miUPGRADE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Upgrade: '%s'\n",
-                           me->value));
-               break;
-           case miURI:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP URI: '%s'\n",
-                           me->value));
-               break;
-           case miVARY:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Vary: '%s'\n",
-                           me->value));
-               break;
-           case miVIA:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Via: '%s'\n",
-                           me->value));
-               break;
-           case miWARNING:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP Warning: '%s'\n",
-                           me->value));
-               break;
-           case miWWW_AUTHENTICATE:
-               HTMIME_TrimDoubleQuotes(me->value);
-               CTRACE((tfp, "HTMIME: PICKED UP WWW-Authenticate: '%s'\n",
-                           me->value));
-               break;
-           default:            /* Should never get here */
-               break;
-           }
-       } else {
+    GET_VALUE:
+       if (c != '\n') {                        /* Not end of line */
            if (me->value_pointer < me->value + VALUE_SIZE - 1) {
                *me->value_pointer++ = c;
                break;
@@ -1651,12 +1718,12 @@
                goto value_too_long;
            }
        }
-       /* Fall through */
+       /* Fall through (if end of line) */
 
     case miJUNK_LINE:
        if (c == '\n') {
-           me->state = miNEWLINE;
            me->fold_state = me->state;
+           me->state = miNEWLINE;
        }
        break;
 
@@ -1728,9 +1795,13 @@
 PRIVATE void HTMIME_free ARGS1(
        HTStream *,     me)
 {
-    if (me->target)
-       (*me->targetClass._free)(me->target);
-    FREE(me);
+    if (me) {
+       FREE(me->location);
+       FREE(me->compression_encoding);
+       if (me->target)
+           (*me->targetClass._free)(me->target);
+       FREE(me);
+    }
 }
 
 /*     End writing
@@ -1739,9 +1810,13 @@
        HTStream *,     me,
        HTError,        e)
 {
-    if (me->target)
-       (*me->targetClass._abort)(me->target, e);
-    FREE(me);
+    if (me) {
+       FREE(me->location);
+       FREE(me->compression_encoding);
+       if (me->target)
+           (*me->targetClass._abort)(me->target, e);
+       FREE(me);
+    }
 }
 
 
@@ -1841,6 +1916,21 @@
        return NULL;
 
     me->net_ascii = YES;
+    return me;
+}
+
+PUBLIC HTStream* HTMIMERedirect ARGS3(
+       HTPresentation *,       pres,
+       HTParentAnchor *,       anchor,
+       HTStream *,             sink)
+{
+    HTStream* me = HTMIMEConvert(pres,anchor, sink);
+    if (!me)
+       return NULL;
+
+    me->pickup_redirection = YES;
+    if (me->targetRep == WWW_DEBUG && sink)
+       me->no_streamstack = YES;
     return me;
 }
 
Index: 2.30/WWW/Library/Implementation/HTStream.h
--- 2.30/WWW/Library/Implementation/HTStream.h Fri, 04 Jun 1999 20:57:16 -0500
+++ 2.30(w)/WWW/Library/Implementation/HTStream.h Sat, 04 Dec 1999 01:59:53 
-0600
@@ -54,6 +54,17 @@
 
 }HTStreamClass;
 
+/*
+
+  Generic Error Stream
+  
+   The Error stream simply signals an error on all output methods.
+   This can be used to stop a stream as soon as data arrives, for
+   example from the network.
+   
+ */
+extern HTStream * HTErrorStream (void);
+
 #endif /* HTSTREAM_H */
 
 /*
Index: 2.30/WWW/Library/Implementation/HTFormat.h
--- 2.30/WWW/Library/Implementation/HTFormat.h Wed, 06 Oct 1999 13:57:53 -0500
+++ 2.30(w)/WWW/Library/Implementation/HTFormat.h Sat, 04 Dec 1999 01:59:56 
-0600
@@ -48,6 +48,14 @@
  */
 #define WWW_PRESENT HTAtom_for("www/present")   /* The user's perception */
 
+#define WWW_DEBUG       HTAtom_for("www/debug")
+/*
+
+   WWW_DEBUG represents the user's perception of debug information, for 
example sent as a
+   HTML document in a HTTP redirection message.
+   
+ */
+
 /*
 
    The message/rfc822 format means a MIME message or a plain text message with 
no MIME
@@ -55,6 +63,11 @@
 
  */
 #define WWW_MIME HTAtom_for("www/mime")         /* A MIME message */
+
+/*
+  For parsing only the header. - kw
+  */
+#define WWW_MIME_HEAD   HTAtom_for("message/x-rfc822-head")
 
 /*
 
Index: 2.30/WWW/Library/Implementation/HTMIME.h
--- 2.30/WWW/Library/Implementation/HTMIME.h Fri, 04 Jun 1999 20:20:33 -0500
+++ 2.30(w)/WWW/Library/Implementation/HTMIME.h Sat, 04 Dec 1999 01:59:59 -0600
@@ -52,6 +52,14 @@
 extern HTStream * HTNetMIME PARAMS((HTPresentation * pres,
                                         HTParentAnchor * anchor,
                                         HTStream * sink));
+/*
+
+  INPUT: Redirection message, parse headers only for Location if present
+
+ */
+extern HTStream * HTMIMERedirect PARAMS((HTPresentation * pres,
+                                        HTParentAnchor * anchor,
+                                        HTStream * sink));
 
 
 /*
Index: 2.30/WWW/Library/Implementation/HTFile.c
--- 2.30/WWW/Library/Implementation/HTFile.c Thu, 25 Nov 1999 07:18:32 -0600
+++ 2.30(w)/WWW/Library/Implementation/HTFile.c Sat, 04 Dec 1999 02:00:03 -0600
@@ -2512,6 +2512,13 @@
 
           } /* end if localname is a directory */
 
+           if (S_ISREG(dir_info.st_mode)) {
+#ifdef INT_MAX
+               if (dir_info.st_size <= INT_MAX)
+#endif
+                   anchor->content_length = dir_info.st_size;
+           }
+
        } /* end if file stat worked */
 
 /* End of directory reading section
Index: 2.30/src/LYCurses.c
--- 2.30/src/LYCurses.c Sat, 04 Dec 1999 00:06:06 -0600
+++ 2.30(w)/src/LYCurses.c Sat, 04 Dec 1999 02:00:38 -0600
@@ -945,7 +945,8 @@
 
 PUBLIC void stop_curses NOARGS
 {
-    echo();
+    if (LYCursesON)
+       echo();
 #ifdef __DJGPP__
 #ifdef WATT32
     _eth_release();
Index: 2.30/src/LYStrings.c
--- 2.30/src/LYStrings.c Sat, 04 Dec 1999 00:06:06 -0600
+++ 2.30(w)/src/LYStrings.c Sat, 04 Dec 1999 02:00:55 -0600
@@ -1080,7 +1080,11 @@
                *t = '\0';
            if (map_string_to_keysym (s, &keysym) >= 0
             && unescape_string(parse, buf)) {
-               CTRACE((tfp, "KEYMAP(DEF) keysym=%#x, seq='%s'\n", keysym, 
buf));
+               if (LYTraceLogFP == 0) {
+                   CTRACE((tfp, "KEYMAP(DEF) keysym=%#x\n", keysym));
+               } else {
+                   CTRACE((tfp, "KEYMAP(DEF) keysym=%#x, seq='%s'\n", keysym, 
buf));
+               }
                return define_key(buf, keysym);
            }
            else {
Index: 2.30/src/HTInit.c
--- 2.30/src/HTInit.c Mon, 25 Oct 1999 09:01:18 -0500
+++ 2.30(w)/src/HTInit.c Sat, 04 Dec 1999 02:00:58 -0600
@@ -75,6 +75,12 @@
 /*
  *  Add our header handlers.
  */
+ HTSetConversion("message/x-http-redirection", "*",
+                                            HTMIMERedirect, 2.0, 0.0, 0.0, 0);
+ HTSetConversion("message/x-http-redirection", "www/present",
+                                            HTMIMERedirect, 2.0, 0.0, 0.0, 0);
+ HTSetConversion("message/x-http-redirection", "www/debug",
+                                            HTMIMERedirect, 1.0, 0.0, 0.0, 0);
  HTSetConversion("www/mime",  "www/present",  HTMIMEConvert, 1.0, 0.0, 0.0, 0);
  HTSetConversion("www/mime",  "www/download", HTMIMEConvert, 1.0, 0.0, 0.0, 0);
  HTSetConversion("www/mime",  "www/source",   HTMIMEConvert, 1.0, 0.0, 0.0, 0);
@@ -83,8 +89,6 @@
 /*
  *  Add our compressed file handlers.
  */
- HTSetConversion("www/compressed", "www/present",
-                                             HTCompressed,   1.0, 0.0, 0.0, 0);
  HTSetConversion("www/compressed", "www/download",
                                              HTCompressed,   1.0, 0.0, 0.0, 0);
  HTSetConversion("www/compressed", "www/present",


reply via email to

[Prev in Thread] Current Thread [Next in Thread]