[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly

[John W. Eaton]

Follow-up Comment #8, bug #54405 (project octave):

We discussed signed vs. unsigned for octave_idx_type in this thread:

http://lists.gnu.org/archive/html/octave-maintainers/2016-11/msg00183.html

decided to go with a signed type.

The check that attempts to ensure that the overall array size is valid given a
set of individual array dimensions is in the function safe_numel in
dim_vector.cc.  Is that check wrong?

I'm not opposed to switching to using size_t for array sizes internally in
Octave as it would make using the STL a bit easier.  But does it really
matter?  Is it worth the effort to switch?


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54405>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/