bug-classpath
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug-classpath] [bug #13532] The charset decoder for UnicodeLittle gives


From: Ito Kazumitsu
Subject: [bug-classpath] [bug #13532] The charset decoder for UnicodeLittle gives wrong results
Date: Mon, 27 Jun 2005 06:34:03 +0000
User-agent: w3m/0.5.1

Follow-up Comment #1, bug #13532 (project classpath):

> Although Sun's document says that UnicodeLittle is with byte-order mark,
> the default byte order of UnicodeLittle should be little endian.
> UnicodeLittle without byte order mark seems to be a common practice.

Seeing the behavior of Sun's JDK, UnicodeLittle with or without byte order
mark should be treated as follows:

  UnicodeLittle with correct byte order mark:
    Ignore the byte order mark and continue assuming the byte order
    to be little endian.

  UnicodeLittle with incorrect byte order mark:
    The byte sequence is malformed.

  UnicodeLittle without byte order mark:
    Continue assuming the byte order to be little endian.

Then the patch will be like this:

--- gnu/java/nio/charset/UnicodeLittle.java.orig        Tue Apr 19 19:12:23 2005
+++ gnu/java/nio/charset/UnicodeLittle.java     Mon Jun 27 14:44:27 2005
@@ -64,7 +64,7 @@
 
   public CharsetDecoder newDecoder ()
   {
-    return new UTF_16Decoder (this, UTF_16Decoder.UNKNOWN_ENDIAN);
+    return new UTF_16Decoder (this, UTF_16Decoder.MAYBE_LITTLE_ENDIAN);
   }
 
   public CharsetEncoder newEncoder ()

--- gnu/java/nio/charset/UTF_16Decoder.java.orig        Tue Apr 19 19:12:23 2005
+++ gnu/java/nio/charset/UTF_16Decoder.java     Mon Jun 27 14:55:04 2005
@@ -54,6 +54,8 @@
   static final int BIG_ENDIAN = 0;
   static final int LITTLE_ENDIAN = 1;
   static final int UNKNOWN_ENDIAN = 2;
+  static final int MAYBE_BIG_ENDIAN = 3;
+  static final int MAYBE_LITTLE_ENDIAN = 4;
 
   private static final char BYTE_ORDER_MARK = 0xFEFF;
   private static final char REVERSED_BYTE_ORDER_MARK = 0xFFFE;
@@ -81,32 +83,44 @@
             byte b2 = in.get ();
 
             // handle byte order mark
-            if (byteOrder == UNKNOWN_ENDIAN)
+            if (byteOrder == UNKNOWN_ENDIAN ||
+                byteOrder == MAYBE_BIG_ENDIAN ||
+                byteOrder == MAYBE_LITTLE_ENDIAN)
               {
-                char c = (char) (((b1 & 0xFF) << 8) | (b2 & 0xFF));
+                char c = (char) (((b1 & 0x00FF) << 8) | (b2 & 0x00FF));
                 if (c == BYTE_ORDER_MARK)
                   {
+                    if (byteOrder == MAYBE_LITTLE_ENDIAN)
+                      {
+                        return CoderResult.malformedForLength (2);
+                      }
                     byteOrder = BIG_ENDIAN;
                     inPos += 2;
                     continue;
                   }
                 else if (c == REVERSED_BYTE_ORDER_MARK)
                   {
+                    if (byteOrder == MAYBE_BIG_ENDIAN)
+                      {
+                        return CoderResult.malformedForLength (2);
+                      }
                     byteOrder = LITTLE_ENDIAN;
                     inPos += 2;
                     continue;
                   }
                 else
                   {
-                    // assume big endian, do not consume bytes,
+                    // assume big or little endian, do not consume bytes,
                     // continue with normal processing
-                    byteOrder = BIG_ENDIAN;
+                    byteOrder = (byteOrder == MAYBE_LITTLE_ENDIAN ?
+                                 LITTLE_ENDIAN : BIG_ENDIAN);
                   }
               }
 
            // FIXME: Change so you only do a single comparison here.
-            char c = byteOrder == BIG_ENDIAN ? (char) ((b1 << 8) | b2)
-                                             : (char) ((b2 << 8) | b1);
+            char c = byteOrder == BIG_ENDIAN ?
+                 (char) (((b1 & 0x00FF) << 8) | (b2 & 0x00FF)) :
+                 (char) (((b2 & 0x00FF) << 8) | (b1 & 0X00FF));
 
             if (0xD800 <= c && c <= 0xDFFF)
               {


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?func=detailitem&item_id=13532>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/





reply via email to

[Prev in Thread] Current Thread [Next in Thread]