help-octave
[Top][All Lists]

## Re: Problem with the floor function

 From: Michael Hanke Subject: Re: Problem with the floor function Date: Tue, 19 Jan 1999 08:30:45 +0100 (MET)

```Hej,

You ran in the interesting world of rounding errors. All computations
in octave (and matlab) are done in double precision IEEE numbers.
They use the base 2 while we use conveniently decimal numbers for
input/output representation. You are computing something like
15/0.01. But 0.01 does not have a finite representation in the binary
system. So you will not obtain 1500 but a rounded result. Your octave
session indicates that it is slightly lower than 1500.

This problem is very common in defining a grid in floating point
numbers:

a=...;
b=...;
h=(b-a)/m;
x=a:h:b;

gives often not the desired result. I am using instead

x=a:h:b+h/2;

in order to avoid a surprise when considering the length of x.

Hope this helps.

Michael

On Tue, 19 Jan 1999, Daniel Tourde wrote:

> Hello,
>
> I have discovered something strange with the floor function. I suspect a
> bug but where ? John W. Eaton would bet that floor is correct, but that
> the display routines in the C/C++ I/O library are doing
> something slightly different when they display values that are very
> near but just less than a representable integer.
>
> octave:1> floor(46.000)
> ans = 46
> octave:2> k = 0.03
> k = 0.030000
> octave:3> kmin=0
> kmin = 0
> octave:4> kstep = 0.01
> kstep = 0.010000
> octave:5> modes=15
> modes = 15
> octave:6> (1+(k-kmin)*modes/kstep)
> ans = 46.000
> octave:7> floor(1+(k-kmin)*modes/kstep)
> ans = 45                                     <-  It should be 46, no ?
> octave:8> (k-kmin)*modes
> ans = 0.45000
> octave:9> (k-kmin)*modes/kstep
> ans = 45.000
> octave:10> floor((k-kmin)*modes/kstep)
> ans = 44                                     <-  It should be 45, no ?
>
> Strange isn't it ?
> Does anyone has an idea of where it comes from and how it could be
> solved ?
>
>
>                                 Daniel
> --
> ***********************************************************************
> Daniel TOURDE                                   E-mail : address@hidden
> The Aeronautical Research Institute of Sweden   Tel : +46 8 55 54 93 44
> P.O. Box 11021 S-161 11 BROMMA, Sweden          Fax : +46 8    25 34 81
> ***********************************************************************
>
>

+---------------------------------------------------------------+
|  Michael Hanke                Royal Institute of Technology   |
|                               S-10044 Stockholm               |
|                               Sweden                          |
+---------------------------------------------------------------+
|  Visiting address:            Lindstedtsvaegen 3              |
|  Phone:                       + (46) (8) 790 6278             |
|  Fax:                         + (46) (8) 790 0930             |