[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow
From: |
Dan Sebald |
Subject: |
[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly |
Date: |
Sun, 29 Jul 2018 23:59:16 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0 |
Follow-up Comment #3, bug #54405 (project octave):
Something else I just remembered: I think that both gcc and VCC have integer
math overflow trap mechanism (-trapv in gcc or something similar).
Also, GNU has some builtin routines that will check for overflow:
https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html
These might be more efficient than the
// Check for overflow.
if (nr != 0 && abs (nc) > abs (std::numeric_limits<octave_idx_type>::max
() / nr))
{
std::cerr << "ERROR OCCURED\n";
error ("fread: dimension too large for Octave's index type");
approach because that division above could be wasteful for small matrix sizes.
That is, C/C++ has no natural language elements for checking overflows, but
GNU might have optimized routines at the machine code level that can easily
check a status flag after the multiplication is done.
But Octave aims for all different platforms in a general sense, so using the
traps or the GNU
bool __builtin_smul_overflow (int a, int b, int *res)
is sort of out.
Otherwise, is there a more efficient pre-multiply check?
log2(nc) > log2(std::numeric_limits<octave_idx_type>::max ()) - log2(nr)
perhaps? The log2(MAX) part is just a constant. log2 might have a really
efficient integer-based counterpart.
Or, one could do
if ((nr > 2^32 || nc > 2^32) && nr != 0
&& nc > (std::numeric_limits<octave_idx_type>::max () / nr))
{
error ("fread: dimension too large");
Meaning that at least one of the nr and nc has to be greater than
2^(log2(MAX)/2) otherwise there will be no multiplication overflow. That is a
very quick way of allowing a very large percentage of the most typical vector
sizes, i.e., fairly small. Only when the dimension sizes start to get big
would a division be done to check for overflow.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54405>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/29
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/29
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/29
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly,
Dan Sebald <=
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/30
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/30
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/30
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, John W. Eaton, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dmitri A. Sergatskov, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31