[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Is this a numerical problem?
From: |
Lukas Reichlin |
Subject: |
Re: Is this a numerical problem? |
Date: |
Thu, 19 Aug 2010 19:22:35 +0200 |
On 19.08.2010, at 18:36, Jordi Gutiérrez Hermoso wrote:
> 2010/8/19 Judd Storrs <address@hidden>:
>> 2010/8/19 Jordi Gutiérrez Hermoso <address@hidden>
>>>
>>> I believe the inconsistency is how 0.999995 is displayed in each with
>>> "format short". Octave displays 0.99999 and Matlab displays 1.0000,
>>> from what I can read in the bug report.
>>
>> I don't understand this comparison. Matlab is displaying four digits after
>> the decimal whereas octave is displaying five. If octave displayed four
>> digits it would also display 1.0000.
>
> It's five digits total. Format short displays five digtis.
>
>> Why is Matlab more correct? The inaccuracy of the Matlab representation is
>> +0.000005 and the inaccuracy of the Octave representation is -0.000005. Why
>> is one "correct" and the other wrong?
>
> Because 5 should round up. Precisely for reasons like this.
>
>> Last time I checked GCC has very excellent binary to decimal conversion
>> routines.
>
> They're not perfect. It's a very fiddly picky thing to worry about.
> Here's a blog post with links to gcc bug reports:
>
>
> http://www.exploringbinary.com/incorrectly-rounded-conversions-in-gcc-and-glibc/
>
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Your link is very interesting. As far as I understood, an 8-bit mantissa
approximates a decimal number with x/256, where x is an 8-bit integer. Now it
might happen that the closest approximation of, let's say, 0.9995 is a binary
value representing a fraction which is slightly smaller than 0.9995. Thus it
gets rounded down when we cut off the last decimal digit, whereas I expected
the opposite, regarding the decimal representation.
Best Regards,
Lukas
- Re: Is this a numerical problem?, (continued)
- Re: Is this a numerical problem?, George Barrick, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Ozzy Lash, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?,
Lukas Reichlin <=
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, George Barrick, 2010/08/19