Re: rem and mod

[Olaf Till]

On Thu, Jun 26, 2014 at 03:21:28PM +0200, José Luis García Pallero wrote:
> 2014-06-26 12:45 GMT+02:00 Paul Soderlind <address@hidden>:
> > Hi all,
> >
> > rem(0.94,0.01) gives the answer 0.01 (in 3.8.1-4 from mxeoctave.osuv.de, but
> > also in other windows binaries), but I had expected to get 0.  mod is
> > similar.
> >
> > The reason seems to be related to that fix(0.94/0.01) = 93, since 0.94/0.01
> > - 94 is evaluated to be -1.4211e-14. Clearly, floating point precision
> > playing tricks. rem(94,1) does the right thing
> >
> > Is this to be expected? (...matlab gives 0 and Octave''s ) Would be good to
> > know. Thanks
> 
> Octave 3.8.1 compiled by myself in Debian also gives the incorrect 0.01 answer

According to function xrem() in liboctave/numeric/lo-mappers.h (tip of
default branch), for

xrem (x, y) and q defined as q := x / y,

there is a case distinction for abs(x) <= 1 or > 1. For abs(x) > 1, q
is adjusted to an integer if its calculated deviation from an integer
is less than roughly the accuracy (epsilon * q) with which q is
represented (by a double in this case), which seems ok. But for abs(x)
<= 1, the assumed accuracy of q seems to be just epsilon, not epsilon
* q; whats the reason for this, or what am I seeing wrong? If the case
abs(x) <= 1 were not treated special, the OPs result probably would be
as expected (not tested).

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature