[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction

[Dan Sebald]

Follow-up Comment #15, bug #54572 (project octave):

Oh, I see.  Yeah, I suspect it is that float-53/int-64 issue again.  Recall,
before that issue manifested as a float index mapped to an integer negative
number (or something like that).  Here, I would guess that the arithmetic
with


  // Compute proper thresholds.
  static const S thmin = compute_threshold (static_cast<S> (min_val ()),
                                            min_val ());
  static const S thmax = compute_threshold (static_cast<S> (max_val ()),
                                            max_val ());
[snip]
  compute_threshold (S val, T orig_val)
  {
    val = std::round (val); // Fool optimizations (maybe redundant)
    // If val is even, but orig_val is odd, we're one unit off.
    if (orig_val % 2 && val / 2 == std::round (val / 2))
      // FIXME: is this always correct?
      val *= (static_cast<S> (1) - (std::numeric_limits<S>::epsilon () / 2));
    return val;
  }


is producing a threshold "thmin" that is one resolution bit on the negative
side of LINT_MIN or std::numeric_limits<int64_t>::min.  No time to investigate
right now.

However, I'm not certain of the above guess because the decrement operator
doesn't appear to compute a limit and it fails:


octave:8> intmin ('int64') - 1
I am in INT_DOUBLE_BINOP_DECL (-, int64)
x = -9223372036854775808
(-y) = -1
MIN THRESH: -9.22337e+18
MAX THRESH: 9.22337e+18
ans = 9223372036854775807
octave:9> x = intmin('int64')
x = -9223372036854775808
octave:10> --x
ans = 9223372036854775807



    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54572>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/