Re: Is this a numerical problem?

[Lukas Reichlin]

Hi Jaroslav

I realized that (in most cases, see below) floats are rounded, not truncated 
such that -4.00004 and -3.99995 are both displayed as -4.0000. Therefore you 
can't tell whether -4.0000 is less or greater than -4.

octave:15> 3.9999
ans =  3.9999
octave:16> 3.99995
ans =  4.0000
octave:17> 0.99995
ans =  0.99995
octave:18> 0.999995
ans =  0.99999  # I'm expecting 1.00000 here
octave:19> 0.9999995
ans =  1.00000
octave:20> 0.999996
ans =  1.00000  # compare this to command 18

Regards,
Lukas

On 19.08.2010, at 11:43, Jaroslav Hajek wrote:

> On Thu, Aug 19, 2010 at 11:31 AM, Lukas Reichlin
> <address@hidden> wrote:
>> Hi Jaroslav
>> 
>> Thanks for your quick reply. # a < -4 is meant as a comment about what I've 
>> learned at elementary school ;-) -4.0000...001 is smaller than -4 while 
>> -3.99999… is greater than -4.
> 
> Okay, you've been taught well :) So, if a number is displayed as
> -4.00000000000000, does it in reality look like -4.0000...001 or
> -3.99999...?
> 
> -- 
> RNDr. Jaroslav Hajek, PhD
> computing expert & GNU Octave developer
> Aeronautical Research and Test Institute (VZLU)
> Prague, Czech Republic
> url: www.highegg.matfyz.cz