[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## Re: Why does this happen

**From**: |
A. Scottedward Hodel |

**Subject**: |
Re: Why does this happen |

**Date**: |
Tue, 13 Oct 1998 10:29:15 -0500 |

>* From address@hidden Tue Oct 13 10:24:37 1998*
>* *
>* *
>* B =*
>* *
>* 7973.60000*
>* 0.96500*
>* 2.71400*
>* *
>* octave:5> B*
>* B =*
>* *
>* 7973.600000000000364*
>* 0.965000000000000*
>* 2.714000000000000*
>* *
>* *
>* Why is B(1) now suddenly 7973.600000000000364 and not*
>* 7973.600000000000000 ?*
Because double precision arithmetic has approx 16 digits of precision,
and because numbers are stored using (usually) base 2, not base 10.
Finite-length base 10 fractions are often repeating numbers in binary.
This is inherent in any floating-point computation.