octave-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #55238] Modulo [mod(X, Y)] of larger values s


From: Rik
Subject: [Octave-bug-tracker] [bug #55238] Modulo [mod(X, Y)] of larger values seems to fail silently to output 0
Date: Sun, 23 Dec 2018 12:21:42 -0500 (EST)
User-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko

Follow-up Comment #8, bug #55238 (project octave):

I wouldn't say flintmax is not the point.  While mod does work for
non-integers, the overwhelming use case is for integers.  Given that mod
satisfies user's expectations as long as a reduced input range is used, I
think it is worth noting that in the documentation.

For reference, Octave maps integer variable classes to the integer mod
operator '%' in C++.  So for integer values greater than 2^53 it is possible
to convert from double to uint64 and still get expected results.  For inputs
of class double, we don't use fmod directly because Octave needs to match
rounding rules for near-integers for Matlab compatibility.  In effect, Octave
does calculate what is described in the documentation


retval = x - floor (x/y) * y


In the case from this code, x/y is still so large (10^26) that there is no
fractional decimal portion that floor can truncate and it is a nop.  Hence,


retval = x - floor (x/y) * y
===
retval = x - (x/y) * y
===
retval = x - x
===
retval = 0


which is Octave is returning.



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?55238>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/




reply via email to

[Prev in Thread] Current Thread [Next in Thread]