Re: precision of floor(a/b)

[Ben Abbott]

On Apr 5, 2010, at 7:52 PM, Tim Rueth wrote:

> If I do floor(6/1), I correctly get 6.  If I do floor(0.6/0.1), I get 5.  If 
> I do floor((0.6/0.1 + 0.00000001)) I correctly get 6.
>  
> I assume this has to do with the precision of the division?  I read in the 
> manual that Octave stores numbers as double precision, so this surprises me.  
> What's the best way to deal with this imprecision so I can always get the 
> correct answer, or is something else going on here?

Will floor(0.6/0.1*(1+eps)) work for you?

Ben