[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Is this a numerical problem?

From: Jaroslav Hajek
Subject: Re: Is this a numerical problem?
Date: Thu, 19 Aug 2010 21:14:22 +0200

2010/8/19 Jordi Gutiérrez Hermoso <address@hidden>:
> 2010/8/19 Judd Storrs <address@hidden>:
>> 2010/8/19 Jordi Gutiérrez Hermoso <address@hidden>
>>> I believe the inconsistency is how 0.999995 is displayed in each with
>>> "format short". Octave displays 0.99999 and Matlab displays 1.0000,
>>> from what I can read in the bug report.
>> I don't understand this comparison. Matlab is displaying four digits after
>> the decimal whereas octave is displaying five. If octave displayed four
>> digits it would also display 1.0000.
> It's five digits total. Format short displays five digtis.
>> Why is Matlab more correct? The inaccuracy of the Matlab representation is
>> +0.000005 and the inaccuracy of the Octave representation is -0.000005. Why
>> is one "correct" and the other wrong?
> Because 5 should round up. Precisely for reasons like this.

But remember that the value is rounded already at the point it is
entered, so it's not exactly what you think it is.

And watch this:

octave:1> x = 0.999995
x =  0.99999
octave:2> x - 0.99999 < 1.0 - x
ans =  1


octave:3> round(1e5*x)/1e5
ans =  1

That's why I invited Lukas to submit a bug report. This may require
some investigation, and ML threads are always in danger of being


RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic

reply via email to

[Prev in Thread] Current Thread [Next in Thread]