[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: precision of calculations
From: |
A. Scottedward Hodel |
Subject: |
Re: precision of calculations |
Date: |
Fri, 23 Jul 1999 07:12:52 -0500 |
double precision arithmetic has about 16 significant digits
(depending on hardware). Hence, if (A-B) should be zero, but
you get
norm(A-B)/norm(A) approximately = 1e-16
you're o.k. The relative error can creep up as problem dimension
and problem condition (how sensitive the results are to small
perturbations in data) increases.
For more information, consult Golub and Van Loan's book,
Matrix Computations, Johns Hopkins University Press, available
in paperback (for the frugal) or hardback (for those who use it
a lot).
A S Hodel Assoc. Prof. Dept Elect Eng, Auburn Univ,AL 36849-5201
On leave at NASA Marshall Space Flight Center (256) 544-1426
Address until 15 Mar 2000:Mail Code TD-55, MSFC, Alabama, 35812
http://www.eng.auburn.edu/~scotte
----------
>From: Robert Butora <address@hidden>
>To: address@hidden
>Subject: precision of calculations
>Date: Fri, Jul 23, 1999, 7:27 AM
>
>Does anybody know how the precision of calculation
>(not output_precision !) is controlled in Octave ?
>
>I've calculated some matrix where I expected results
>to be 1 or 0. Instead of 0 I've got values in order
>of 1e-16 ... 1e-18.
>
>Should I chase an error in my calculation or does it
>have something to do with precision of calc. ?
>
>Could anybody give some general thoughts on precision in Octave ?
>
>Thanks
>Robert
>
>
>
>---------------------------------------------------------------------
>Octave is freely available under the terms of the GNU GPL. To ensure
>that development continues, see www.che.wisc.edu/octave/giftform.html
>Instructions for unsubscribing: www.che.wisc.edu/octave/archive.html
>---------------------------------------------------------------------
>
---------------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL. To ensure
that development continues, see www.che.wisc.edu/octave/giftform.html
Instructions for unsubscribing: www.che.wisc.edu/octave/archive.html
---------------------------------------------------------------------