[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Is this a numerical problem?
From: |
Jordi Gutiérrez Hermoso |
Subject: |
Re: Is this a numerical problem? |
Date: |
Thu, 19 Aug 2010 11:36:56 -0500 |
2010/8/19 Judd Storrs <address@hidden>:
> 2010/8/19 Jordi Gutiérrez Hermoso <address@hidden>
>>
>> I believe the inconsistency is how 0.999995 is displayed in each with
>> "format short". Octave displays 0.99999 and Matlab displays 1.0000,
>> from what I can read in the bug report.
>
> I don't understand this comparison. Matlab is displaying four digits after
> the decimal whereas octave is displaying five. If octave displayed four
> digits it would also display 1.0000.
It's five digits total. Format short displays five digtis.
> Why is Matlab more correct? The inaccuracy of the Matlab representation is
> +0.000005 and the inaccuracy of the Octave representation is -0.000005. Why
> is one "correct" and the other wrong?
Because 5 should round up. Precisely for reasons like this.
> Last time I checked GCC has very excellent binary to decimal conversion
> routines.
They're not perfect. It's a very fiddly picky thing to worry about.
Here's a blog post with links to gcc bug reports:
http://www.exploringbinary.com/incorrectly-rounded-conversions-in-gcc-and-glibc/
- Re: Is this a numerical problem?, (continued)
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Miguel Bazdresch, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, George Barrick, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Ozzy Lash, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?,
Jordi Gutiérrez Hermoso <=
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, George Barrick, 2010/08/19