[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

NSNumberFormatter give bad results for NSNumber numberWithDouble

From: Ron Lentjes
Subject: NSNumberFormatter give bad results for NSNumber numberWithDouble
Date: Sat, 1 Aug 2009 22:17:02 -0700 (PDT)

NSNumber* num = [NSNumber numberWithDouble: 2.0 / 3.0];
NSNumberFormatter* form = [[NSNumberFormatter alloc] init];
[form setPositiveFormat: @"#.000(15-zeros)000"];
NSString* s = [form stringFromNumber: num];
printf ("s (%s)\n", [s cString]);
s (00.666667E14000000)
I don't know why 00 before dec pnt, but after looking at code, the conversion from double to NSDecimal is done using a printf  "%g" which makes it only show 6 past dec pnt and will flip into E format but why E+14?
This is 0.666666666...
One possible semi-fix is to use %38.38g (or whatever max is in NSDecimal).
But do note that the more accurate fix would be to start from none rounded digits. So the format would have to be something like %1024.1024 (or what ever to get all 53 bits translated to most possible non-rounded digits then chop and round LS-digit (or however NSDecimal is supposed to work) - BUT SEE BELOW FOR WHAT CAN WORK IN BOLD.
But as is, this language fails the most basic tests I have for languages. Of couse you can always revert to C / C++ printf instead, but I am trying to spur on a fix to GNU object-C / C++ for everyone.
Also what is strange:
123456789 as and integer in NSNumber becomes 12,345,7E8. I tried all kinds of ### formatting but can't get it to work. Last attempt for this result is #(10 of them)#,###".
gnustep-make 2.0.7
GNU obj-c 4.1.2
GNU obj-c++ 4.1.2
gnustep-startup 0.22.0
(I notice 0.23.0 now, but can't find any information on what you guys fixed between 0.22 and 0.23 ??)
For now I will avoid any use of NSNumberFormatter, and just use printf () for now.
Out of 40+ languages only Smalltalk and Objective-C/C++ have serious formatting issues.
Since I have a vested interest in languages, I look forward to getting this fixed.
Email me back for any suggestions on how GNU Smalltalk and Objective-C/C++ can be fixed. Perhaps add to smalltalk a printf-like facility:
add to String (a class ver and obj ver):
format: '%15.8f' with: dVal1
format: '%3d %3d %3d' with: dVal1 with: dVal2 with: dVal3
And fix Objective-C/C++ class.
Common Issue to get above working is to reliably get the raw digits required.
This is something all GNU langauges should have:
C: GetDoubleDigits (double dVal, char* buf, int nSigDigitCount, bool bRounded, bool& bIsNegative)
C++: string GetDoubleDigits (double dVal, int nSigDigitCount, bool bRounded...)
Smalltalk: Tack onto some class: getDoubleDigits: dVal sigDigitCount: nSigDigitCount ...
Objective-C/C++: perhaps just a function NSGetDoubleDigits... and/or
C's ecvt() funtion rounds so you have to know how many sig digits you want FIRST!
which is not good for loading NSDecimal. C's printf (%760e) does not round because you 
start having 0's at that point (for double's at E-324), so you can truncate that to 38 sig
digits for NSDecimal's struct. Just do a couple double chks on this and make some kind of routine. And perhaps you guys know of yet some other internal routines available.
Note: for windows, you can't do this. printf fails after I think %512e and crashes, and besides, it only puts I think only 16 most sig digits and then all zeros - totally useless there - and the last digit is rounded up!!! Crap. But again, perhaps we can find a solution for Window's as well.
One other way I just about forgot - You can get the mantissa and exp via C's frexp()
but then you'll have to do bignumber mul/div by 2 in base 10 then chop the result to fit (and round by looking at digit past LS-decimal digit. (This method can be used with Windows or Linux).
Then all languages can use this to make anything else from it. It is very imporant to get either rounded or not rounded according to needs. You can't round twice!!!
And what kind of rounding? none/half-up/half-even/other?
For objective C/C++, once you have the digits you can then copy them into NSDecimal's
struct properly, without loosing precision (store upto 38 sig digits there but don't round at that point - only round when formatting.)
If I get time, I will try to upload at least a partial suggestion fix.
Ron Lentjes

Access Yahoo!7 Mail on your mobile. Anytime. Anywhere. Show me how.
reply via email to

[Prev in Thread] Current Thread [Next in Thread]