[cp-patches] Report #4742: the patch is easy to make, but should we really fix that bug?

[Meskauskas Audrius]

The bug #4742 contains a comprehensive test, verifying the work of
StreamTokenizer by parsing randomly generated data under 
randomly set options. It reveals the single difference
between the Sun and Classpath implementations. 

If the numeric character ('0'..'9') obtains a specific status of
comment, whitespace or some other character, Sun's implementation
only treats it as such if found standing alone or in the beginning
of the multi - character number. If the number parsing has once
started, the digits in that number are trated as digits regardless
of they spacific status. 

For example, after calling 
whitespaceChars('1', '1'), it reads "121" as 21 while 
Classpatch implementation reads it as "2", of course.

It is a piece of cake to make a patch for reproducing this
behavior. The StreamTokenizer has many Mauve tests 
and it is easy to ensure this does not introduce any 
obvious regressions.

The question is, to fix of not to fix? A bug or not a bug?
In any case, 4742 can be closed either as "not a bug" or as
"fixed".

P.S. The current Classpath implementation also reads
121 in "Sun style" (as 21) if parseNumbers() are called AFTER
and not before the whitespaceChars('1', '1'). 
Further work would be needed to make parsing independent 
from the order, in that the properties were set.

Audrius Meskauskas

StreamTokenizer.java.diff
Description: Binary data