[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Is this a numerical problem?
From: |
Jaroslav Hajek |
Subject: |
Re: Is this a numerical problem? |
Date: |
Thu, 19 Aug 2010 13:16:48 +0200 |
On Thu, Aug 19, 2010 at 12:52 PM, Lukas Reichlin
<address@hidden> wrote:
> Hi Jaroslav
>
> I realized that (in most cases, see below) floats are rounded, not truncated
> such that -4.00004 and -3.99995 are both displayed as -4.0000. Therefore you
> can't tell whether -4.0000 is less or greater than -4.
>
> octave:15> 3.9999
> ans = 3.9999
> octave:16> 3.99995
> ans = 4.0000
> octave:17> 0.99995
> ans = 0.99995
> octave:18> 0.999995
> ans = 0.99999 # I'm expecting 1.00000 here
> octave:19> 0.9999995
> ans = 1.00000
> octave:20> 0.999996
> ans = 1.00000 # compare this to command 18
>
> Regards,
> Lukas
>
Exactly. Which brings us back to your observation:
octave:1> format long
octave:2> a = (10 * rem (-12/5, 1))
a = -4.00000000000000
octave:3> a - -4.00000000000000
ans = 8.88178419700125e-16
octave:4> -3.99999999999999 - a
ans = 9.32587340685131e-15
which is an explanation.
--
RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz
- Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Message not available
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Sergei Steshenko, 2010/08/19
- Re: Is this a numerical problem?,
Jaroslav Hajek <=
- Re: Is this a numerical problem?, Miguel Bazdresch, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19