[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: RFA/PATCH gtest - NSNumber float != double

From: David Ayers
Subject: Re: RFA/PATCH gtest - NSNumber float != double
Date: Wed, 22 Jan 2003 10:09:58 +0100
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3a) Gecko/20021212

Alexander Malmberg wrote:

David Ayers wrote:
Hmm, I'll take your word  for it.  I just recalled back in the dark ages
someone told me (or I read it somewhere) that 25.1 would somehow be
represented as 2.51 * 10^1 and that a float or double only stored the
mantissa and exponent which shouldn't result in any discrepancies here.

It does, but in base 2, and the fraction 25.1/16 can't be represented
exactly in base 2; you get 2^4 * 1.10010 0011 0011 0011 ...

and verified with OPENSTEP 4.2. The "patch" is withdrawn. I apparently forgot the part about base for floats and doubles also being 2. But I guess it is probably the only way to handle these numbers somehow efficiently for (floating point) processors.

Thanks for the lesson :-)


reply via email to

[Prev in Thread] Current Thread [Next in Thread]