[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly

[Dan Sebald]

Follow-up Comment #15, bug #54405 (project octave):

@Rik, I don't think we can move the overflow test within get_size() because
there is no multiplication done there.  But as can be seen in the new
changesets and some BIST, I added a warning within get_size:

octave:5> x = fread (fid, [2^53-1, 2^53], 'single');
warning: fread: possible loss of resolution converting from float to
octave_idx_type
error: fread: dimension too large for Octave's index type

It's only the second size value of 2^53 that causes a warning.  I based the
logic for the choice on the following result:


octave:5> 2^53 > 2^53 - 1
ans = 1
octave:6> 2^53 + 1 > 2^53


Once again, is this necessary?  Probably not, I like it though because it adds
a bit of clarity.  In the future this might be addressed, say, more precisely
in the interpreter phase.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54405>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/