[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Problem with the floor function
From: |
Michael Hanke |
Subject: |
Re: Problem with the floor function |
Date: |
Tue, 19 Jan 1999 08:30:45 +0100 (MET) |
Hej,
You ran in the interesting world of rounding errors. All computations
in octave (and matlab) are done in double precision IEEE numbers.
They use the base 2 while we use conveniently decimal numbers for
input/output representation. You are computing something like
15/0.01. But 0.01 does not have a finite representation in the binary
system. So you will not obtain 1500 but a rounded result. Your octave
session indicates that it is slightly lower than 1500.
This problem is very common in defining a grid in floating point
numbers:
a=...;
b=...;
h=(b-a)/m;
x=a:h:b;
gives often not the desired result. I am using instead
x=a:h:b+h/2;
in order to avoid a surprise when considering the length of x.
Hope this helps.
Michael
On Tue, 19 Jan 1999, Daniel Tourde wrote:
> Hello,
>
> I have discovered something strange with the floor function. I suspect a
> bug but where ? John W. Eaton would bet that floor is correct, but that
> the display routines in the C/C++ I/O library are doing
> something slightly different when they display values that are very
> near but just less than a representable integer.
>
> octave:1> floor(46.000)
> ans = 46
> octave:2> k = 0.03
> k = 0.030000
> octave:3> kmin=0
> kmin = 0
> octave:4> kstep = 0.01
> kstep = 0.010000
> octave:5> modes=15
> modes = 15
> octave:6> (1+(k-kmin)*modes/kstep)
> ans = 46.000
> octave:7> floor(1+(k-kmin)*modes/kstep)
> ans = 45 <- It should be 46, no ?
> octave:8> (k-kmin)*modes
> ans = 0.45000
> octave:9> (k-kmin)*modes/kstep
> ans = 45.000
> octave:10> floor((k-kmin)*modes/kstep)
> ans = 44 <- It should be 45, no ?
>
> Strange isn't it ?
> Does anyone has an idea of where it comes from and how it could be
> solved ?
>
> Thanks in advance
>
> Daniel
> --
> ***********************************************************************
> Daniel TOURDE E-mail : address@hidden
> The Aeronautical Research Institute of Sweden Tel : +46 8 55 54 93 44
> P.O. Box 11021 S-161 11 BROMMA, Sweden Fax : +46 8 25 34 81
> ***********************************************************************
>
>
+---------------------------------------------------------------+
| Michael Hanke Royal Institute of Technology |
| NADA |
| S-10044 Stockholm |
| Sweden |
+---------------------------------------------------------------+
| Visiting address: Lindstedtsvaegen 3 |
| Phone: + (46) (8) 790 6278 |
| Fax: + (46) (8) 790 0930 |
| Email: address@hidden |
| address@hidden |
+---------------------------------------------------------------+