[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## [Axiom-developer] [#167 Infinite Floats Domain]

**From**: |
wyscc |

**Subject**: |
[Axiom-developer] [#167 Infinite Floats Domain] |

**Date**: |
Mon, 13 Jun 2005 09:13:07 -0500 |

Changes
http://page.axiom-developer.org/zope/mathaction/167InfiniteFloatsDomain/diff
--
??changed:
-Most people would expect the answer of 'z-x' to be '0.16887242 E-20' but this
ignores the fact that the display is converted from an internal binary
representation to a decimal one. During the conversion there is truncation
error (think of 0.1 in base 3 converted to decimal 0.3333... with output at any
finite precision). So 'x' is not what it seems, and neither is 'z'. Below, the
constants are converted to binary internally before computation, at a higher
precision than x (resp. z) to bring out the differences. We can now see that
'x' is larger than 'z'. So Axiom is correct and the expectated answer is wrong.
Most people would expect the answer of 'z-x' to be '0.16887242 E-20' but this
ignores the fact that the display is converted from an internal binary
representation to a decimal one. During the conversion there is truncation
error (think of 0.1 in base 3 converted to decimal 0.3333... with output at any
finite precision). So 'x' is not what it seems, and neither is 'z'. Below, the
constants are converted to binary internally before computation, at a higher
precision than x (resp. z) to bring out the differences. We can now see that
'x' is larger than 'z'. So Axiom is correct and the expected answer is wrong.
--
forwarded from http://page.axiom-developer.org/zope/mathaction/address@hidden

**[Axiom-developer] [#167 Infinite Floats Domain]**,
*wyscc* **<=**