[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Is this a numerical problem?
From: |
Jaroslav Hajek |
Subject: |
Re: Is this a numerical problem? |
Date: |
Thu, 19 Aug 2010 14:55:17 +0200 |
On Thu, Aug 19, 2010 at 2:32 PM, Lukas Reichlin
<address@hidden> wrote:
>
> On 19.08.2010, at 13:16, Jaroslav Hajek wrote:
>
>> On Thu, Aug 19, 2010 at 12:52 PM, Lukas Reichlin
>> <address@hidden> wrote:
>>> Hi Jaroslav
>>>
>>> I realized that (in most cases, see below) floats are rounded, not
>>> truncated such that -4.00004 and -3.99995 are both displayed as -4.0000.
>>> Therefore you can't tell whether -4.0000 is less or greater than -4.
>>>
>>> octave:15> 3.9999
>>> ans = 3.9999
>>> octave:16> 3.99995
>>> ans = 4.0000
>>> octave:17> 0.99995
>>> ans = 0.99995
>>> octave:18> 0.999995
>>> ans = 0.99999 # I'm expecting 1.00000 here
>>> octave:19> 0.9999995
>>> ans = 1.00000
>>> octave:20> 0.999996
>>> ans = 1.00000 # compare this to command 18
>>>
>>> Regards,
>>> Lukas
>>>
>>
>> Exactly. Which brings us back to your observation:
>>
>> octave:1> format long
>> octave:2> a = (10 * rem (-12/5, 1))
>> a = -4.00000000000000
>> octave:3> a - -4.00000000000000
>> ans = 8.88178419700125e-16
>> octave:4> -3.99999999999999 - a
>> ans = 9.32587340685131e-15
>>
>> which is an explanation.
>>
>> --
>> RNDr. Jaroslav Hajek, PhD
>> computing expert & GNU Octave developer
>> Aeronautical Research and Test Institute (VZLU)
>> Prague, Czech Republic
>> url: www.highegg.matfyz.cz
>
> Thanks for your patience. I consider the display of result 54 as an
> inconsistency or a (cosmetic) bug because round (0.5) is 1. Rounding up
> starts at 0.50, not at 0.51.
> (Only the first digit after truncation is significant for rounding)
>
> octave:57> 0.999996
> ans = 1.00000
> octave:53> 0.999995
> ans = 0.99999 # this should be 1.00000 like the previous result
> octave:54> 0.9999951
> ans = 1.00000
> octave:55> 0.9999950
> ans = 0.99999 # this should be rounded up too
> octave:56> 0.9999949
> ans = 0.99999
>
>
> Results from the "dark side":
>>> 0.99995
>
> ans =
>
> 1.0000 % this is correct
>
>>> 0.999949
>
> ans =
>
> 0.9999
>
>>> 0.999950
>
> ans =
>
> 1.0000
>
>>> 0.999951
>
> ans =
>
> 1.0000
>
>>> 0.99996
>
> ans =
>
> 1.0000
>
>>>
>
> Cheers,
> Lukas
>
>
If you think you've found a bug, please report it to the tracker.
--
RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz
- Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Message not available
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Sergei Steshenko, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Miguel Bazdresch, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?,
Jaroslav Hajek <=
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, George Barrick, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Ozzy Lash, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19