|
From: | John W. Eaton |
Subject: | Re: Question re: floor of integer division |
Date: | Tue, 25 Feb 2014 01:19:20 -0500 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131005 Icedove/17.0.9 |
On 02/24/2014 09:42 AM, Rhys Ulerich wrote:
I must be missing something, but I don't understand why floor(i/N) behaves differently than floor(i/int32(N)) for N = 128 in the sample below... octave:1> i = 0:639; octave:2> S = 5; octave:3> N = 128; octave:4> qinv = mod(i,N)*S + floor(i/N); min(qinv), max(qinv) ans = 0 ans = 639 octave:5> qinv = mod(i,N)*S + floor(i/int32(N)); min(qinv), max(qinv) ans = 0 ans = 640 This is atop vection 3.6.4.
In Octave, as in Matlab, constants like 0:639, 5, and 128 are stored as double precision floating point values, not as integers. So i/N is performed in floating point but i/int32(N) is performed with the (strange) Matlab rules for integer division. First the division is performed (mathematically; approximated with floating point arithmetic) and the result is rounded to the nearest value when converting back to an integer result.
We didn't invent these rules for Octave. We must follow them for compatibility with Matlab.
jwe -- Follow me on twitter: @theoctaveguy
[Prev in Thread] | Current Thread | [Next in Thread] |