octave-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #46533] numel() returns negative values for la


From: Nicholas Jankowski
Subject: [Octave-bug-tracker] [bug #46533] numel() returns negative values for large diagonal matrices
Date: Fri, 04 Dec 2015 22:36:59 +0000
User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0

Follow-up Comment #1, bug #46533 (project octave):

a quick check in both octave and matlab shows that numel output is a double.
so is the issue just that the row*column step is starting with uint32 numbers
and trying to keeping it that way? knowing that numel should be able to get
bigger than the index limit, can numel be patched to internally class change
to double before whatever arithmetic is causing a problem so this becomes a
non-issue? it's coming out as a double... does it matter where the class
change happens?

(I did play with matlab 2015b and created a 10000x10000x100x100 array of
logicals.  numel(x) output 1e11 without error)

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?46533>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




reply via email to

[Prev in Thread] Current Thread [Next in Thread]