[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## Re: trying to understand why I get 2.2204e-16 rather than 0

**From**: |
Maynard Wright |

**Subject**: |
Re: trying to understand why I get 2.2204e-16 rather than 0 |

**Date**: |
Fri, 05 Aug 2016 22:16:43 -0700 |

**User-agent**: |
KMail/4.13.3 (Linux/3.16.0-30-generic; KDE/4.13.3; i686; ; ) |

On Wednesday, August 03, 2016 08:59:15 PM Dmitri A. Sergatskov wrote:
>* On Wed, Aug 3, 2016 at 7:41 PM, naryana.shankara <address@hidden*
>* > wrote:*
>* > *
>* > If I enter cot (pi/4)-1 I get 2.2204e-16*
>* > From my trig days I remember cot (pi/4) is 1 and 1-1 is 0*
>* > Octave also confirms cot (pi/4) is 1 and 1-1 is 0*
>* *
>* octave:6> format hex*
>* octave:7> 1*
>* ans = 3ff0000000000000*
>* octave:8> cot(pi/4)*
>* *
>* ans = 3ff0000000000001*
>* *
>* > so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?*
>* *
>* https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html*
>* *
>* *
>* Dmitri.*
Although cot(pi/4)-1 returns 0, tan(pi/4)-1 returns -1.11022e-16. One might
therefore suspect that cot() is not obtained by taking the inverse of tan(),
but 1 / (tan(pi/4))-1 returns 0, so the macro cot(x) = 1 / tan(x) may be used,
although I haven't checked.
To see exactly why these anomalies occur, one would have to look into the
floating point representations of the numbers involved and the procedure used
to compute tan(). The references given by others in this thread should be
helpful in pointing out the limitations of representing numbers in floating
point format.
Maynard Wright