[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug-classpath] [bug #13532] The charset decoder for UnicodeLittle gives
From: |
Ito Kazumitsu |
Subject: |
[bug-classpath] [bug #13532] The charset decoder for UnicodeLittle gives wrong results |
Date: |
Mon, 27 Jun 2005 06:34:03 +0000 |
User-agent: |
w3m/0.5.1 |
Follow-up Comment #1, bug #13532 (project classpath):
> Although Sun's document says that UnicodeLittle is with byte-order mark,
> the default byte order of UnicodeLittle should be little endian.
> UnicodeLittle without byte order mark seems to be a common practice.
Seeing the behavior of Sun's JDK, UnicodeLittle with or without byte order
mark should be treated as follows:
UnicodeLittle with correct byte order mark:
Ignore the byte order mark and continue assuming the byte order
to be little endian.
UnicodeLittle with incorrect byte order mark:
The byte sequence is malformed.
UnicodeLittle without byte order mark:
Continue assuming the byte order to be little endian.
Then the patch will be like this:
--- gnu/java/nio/charset/UnicodeLittle.java.orig Tue Apr 19 19:12:23 2005
+++ gnu/java/nio/charset/UnicodeLittle.java Mon Jun 27 14:44:27 2005
@@ -64,7 +64,7 @@
public CharsetDecoder newDecoder ()
{
- return new UTF_16Decoder (this, UTF_16Decoder.UNKNOWN_ENDIAN);
+ return new UTF_16Decoder (this, UTF_16Decoder.MAYBE_LITTLE_ENDIAN);
}
public CharsetEncoder newEncoder ()
--- gnu/java/nio/charset/UTF_16Decoder.java.orig Tue Apr 19 19:12:23 2005
+++ gnu/java/nio/charset/UTF_16Decoder.java Mon Jun 27 14:55:04 2005
@@ -54,6 +54,8 @@
static final int BIG_ENDIAN = 0;
static final int LITTLE_ENDIAN = 1;
static final int UNKNOWN_ENDIAN = 2;
+ static final int MAYBE_BIG_ENDIAN = 3;
+ static final int MAYBE_LITTLE_ENDIAN = 4;
private static final char BYTE_ORDER_MARK = 0xFEFF;
private static final char REVERSED_BYTE_ORDER_MARK = 0xFFFE;
@@ -81,32 +83,44 @@
byte b2 = in.get ();
// handle byte order mark
- if (byteOrder == UNKNOWN_ENDIAN)
+ if (byteOrder == UNKNOWN_ENDIAN ||
+ byteOrder == MAYBE_BIG_ENDIAN ||
+ byteOrder == MAYBE_LITTLE_ENDIAN)
{
- char c = (char) (((b1 & 0xFF) << 8) | (b2 & 0xFF));
+ char c = (char) (((b1 & 0x00FF) << 8) | (b2 & 0x00FF));
if (c == BYTE_ORDER_MARK)
{
+ if (byteOrder == MAYBE_LITTLE_ENDIAN)
+ {
+ return CoderResult.malformedForLength (2);
+ }
byteOrder = BIG_ENDIAN;
inPos += 2;
continue;
}
else if (c == REVERSED_BYTE_ORDER_MARK)
{
+ if (byteOrder == MAYBE_BIG_ENDIAN)
+ {
+ return CoderResult.malformedForLength (2);
+ }
byteOrder = LITTLE_ENDIAN;
inPos += 2;
continue;
}
else
{
- // assume big endian, do not consume bytes,
+ // assume big or little endian, do not consume bytes,
// continue with normal processing
- byteOrder = BIG_ENDIAN;
+ byteOrder = (byteOrder == MAYBE_LITTLE_ENDIAN ?
+ LITTLE_ENDIAN : BIG_ENDIAN);
}
}
// FIXME: Change so you only do a single comparison here.
- char c = byteOrder == BIG_ENDIAN ? (char) ((b1 << 8) | b2)
- : (char) ((b2 << 8) | b1);
+ char c = byteOrder == BIG_ENDIAN ?
+ (char) (((b1 & 0x00FF) << 8) | (b2 & 0x00FF)) :
+ (char) (((b2 & 0x00FF) << 8) | (b1 & 0X00FF));
if (0xD800 <= c && c <= 0xDFFF)
{
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13532>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/