help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Question re: floor of integer division


From: John W. Eaton
Subject: Re: Question re: floor of integer division
Date: Tue, 25 Feb 2014 01:19:20 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131005 Icedove/17.0.9

On 02/24/2014 09:42 AM, Rhys Ulerich wrote:
I must be missing something, but I don't understand why floor(i/N)
behaves differently than floor(i/int32(N)) for N = 128 in the sample
below...

octave:1> i = 0:639;
octave:2> S = 5;
octave:3> N = 128;
octave:4> qinv = mod(i,N)*S + floor(i/N); min(qinv), max(qinv)
ans = 0
ans =  639
octave:5> qinv = mod(i,N)*S + floor(i/int32(N)); min(qinv), max(qinv)
ans = 0
ans = 640

This is atop vection 3.6.4.

In Octave, as in Matlab, constants like 0:639, 5, and 128 are stored as double precision floating point values, not as integers. So i/N is performed in floating point but i/int32(N) is performed with the (strange) Matlab rules for integer division. First the division is performed (mathematically; approximated with floating point arithmetic) and the result is rounded to the nearest value when converting back to an integer result.

We didn't invent these rules for Octave. We must follow them for compatibility with Matlab.

jwe

--
Follow me on twitter: @theoctaveguy



reply via email to

[Prev in Thread] Current Thread [Next in Thread]