[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## precision of calculations

**From**: |
Robert Butora |

**Subject**: |
precision of calculations |

**Date**: |
Fri, 23 Jul 1999 15:27:44 +0300 |

Does anybody know how the precision of calculation
(not output_precision !) is controlled in Octave ?
I've calculated some matrix where I expected results
to be 1 or 0. Instead of 0 I've got values in order
of 1e-16 ... 1e-18.
Should I chase an error in my calculation or does it
have something to do with precision of calc. ?
Could anybody give some general thoughts on precision in Octave ?
Thanks
Robert
---------------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL. To ensure
that development continues, see www.che.wisc.edu/octave/giftform.html
Instructions for unsubscribing: www.che.wisc.edu/octave/archive.html
---------------------------------------------------------------------

**precision of calculations**,
*Robert Butora* **<=**