[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NSNumberFormatter give bad results for NSNumber numberWithDouble

From: Fred Kiefer
Subject: Re: NSNumberFormatter give bad results for NSNumber numberWithDouble
Date: Sun, 02 Aug 2009 19:48:44 +0200
User-agent: Thunderbird (X11/20090605)

One problem here could be that you use an NSNumber. This should worl but
currently we are using a horrible conversion here. We convert the
NSNumber into an NSDecimalNumber via an NSString and then use
NSDecimalNumber for the formating. I wouldn't be too surprised to learn
that this conversion is going wrong. Could you please try again with an
NSDecimalNumber (just create NSDecimalNumbers for 2 and 3 (as Integers)
and divide them.


Ron Lentjes schrieb:
> Hi GNU,
> NSNumber* num = [NSNumber numberWithDouble: 2.0 / 3.0];
> ...
> NSNumberFormatter* form = [[NSNumberFormatter alloc] init];
> ...
> [form setPositiveFormat: @"#.000(15-zeros)000"];
> ...
> NSString* s = [form stringFromNumber: num];
> ...
> printf ("s (%s)\n", [s cString]);
> s (00.666667E14000000)
> I don't know why 00 before dec pnt, but after looking at code, the conversion 
> from double to NSDecimal is done using a printf  "%g" which makes it only 
> show 6 past dec pnt and will flip into E format but why E+14? 
> This is 0.666666666... 
> One possible semi-fix is to use %38.38g (or whatever max is in NSDecimal).
> But do note that the more accurate fix would be to start from none rounded 
> digits. So the format would have to be something like %1024.1024 (or what 
> ever to get all 53 bits translated to most possible non-rounded digits then 
> chop and round LS-digit (or however NSDecimal is supposed to work) - BUT SEE 
> But as is, this language fails the most basic tests I have for languages. Of 
> couse you can always revert to C / C++ printf instead, but I am trying to 
> spur on a fix to GNU object-C / C++ for everyone.
> Also what is strange:
> 123456789 as and integer in NSNumber becomes 12,345,7E8. I tried all kinds of 
> ### formatting but can't get it to work. Last attempt for this result is #(10 
> of them)#,###".
> Versions:
> gnustep-make 2.0.7
> GNU obj-c 4.1.2
> GNU obj-c++ 4.1.2
> gnustep-startup 0.22.0
> (I notice 0.23.0 now, but can't find any information on what you guys fixed 
> between 0.22 and 0.23 ??)
> For now I will avoid any use of NSNumberFormatter, and just use printf () for 
> now.
> Out of 40+ languages only Smalltalk and Objective-C/C++ have serious 
> formatting issues.
> Since I have a vested interest in languages, I look forward to getting this 
> fixed.
> Email me back for any suggestions on how GNU Smalltalk and Objective-C/C++ 
> can be fixed. Perhaps add to smalltalk a printf-like facility:
> add to String (a class ver and obj ver):
> format: '%15.8f' with: dVal1
> format: '%3d %3d %3d' with: dVal1 with: dVal2 with: dVal3
> And fix Objective-C/C++ class.
> Common Issue to get above working is to reliably get the raw digits required.
> This is something all GNU langauges should have:
> C: GetDoubleDigits (double dVal, char* buf, int nSigDigitCount, bool 
> bRounded, bool& bIsNegative)
> C++: string GetDoubleDigits (double dVal, int nSigDigitCount, bool 
> bRounded...)
> Smalltalk: Tack onto some class: getDoubleDigits: dVal sigDigitCount: 
> nSigDigitCount ...
> Objective-C/C++: perhaps just a function NSGetDoubleDigits... and/or
> C's ecvt() funtion rounds so you have to know how many sig digits you want 
> which is not good for loading NSDecimal. C's printf (%760e) does not round 
> because you 
> start having 0's at that point (for double's at E-324), so you can truncate 
> that to 38 sig
> digits for NSDecimal's struct. Just do a couple double chks on this and make 
> some kind of routine. And perhaps you guys know of yet some other internal 
> routines available.
> Note: for windows, you can't do this. printf fails after I think %512e and 
> crashes, and besides, it only puts I think only 16 most sig digits and then 
> all zeros - totally useless there - and the last digit is rounded up!!! Crap. 
> But again, perhaps we can find a solution for Window's as well.
> One other way I just about forgot - You can get the mantissa and exp via C's 
> frexp()
> but then you'll have to do bignumber mul/div by 2 in base 10 then chop the 
> result to fit (and round by looking at digit past LS-decimal digit. (This 
> method can be used with Windows or Linux).
> Then all languages can use this to make anything else from it. It is very 
> imporant to get either rounded or not rounded according to needs. You can't 
> round twice!!!
> And what kind of rounding? none/half-up/half-even/other?
> For objective C/C++, once you have the digits you can then copy them into 
> NSDecimal's
> struct properly, without loosing precision (store upto 38 sig digits there 
> but don't round at that point - only round when formatting.)
> If I get time, I will try to upload at least a partial suggestion fix.
> Cheers,
> Ron Lentjes

reply via email to

[Prev in Thread] Current Thread [Next in Thread]