[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in n

From: Dan Sebald
Subject: [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction
Date: Tue, 28 Aug 2018 16:15:17 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Follow-up Comment #15, bug #54572 (project octave):

Oh, I see.  Yeah, I suspect it is that float-53/int-64 issue again.  Recall,
before that issue manifested as a float index mapped to an integer negative
number (or something like that).  Here, I would guess that the arithmetic

  // Compute proper thresholds.
  static const S thmin = compute_threshold (static_cast<S> (min_val ()),
                                            min_val ());
  static const S thmax = compute_threshold (static_cast<S> (max_val ()),
                                            max_val ());
  compute_threshold (S val, T orig_val)
    val = std::round (val); // Fool optimizations (maybe redundant)
    // If val is even, but orig_val is odd, we're one unit off.
    if (orig_val % 2 && val / 2 == std::round (val / 2))
      // FIXME: is this always correct?
      val *= (static_cast<S> (1) - (std::numeric_limits<S>::epsilon () / 2));
    return val;

is producing a threshold "thmin" that is one resolution bit on the negative
side of LINT_MIN or std::numeric_limits<int64_t>::min.  No time to investigate
right now.

However, I'm not certain of the above guess because the decrement operator
doesn't appear to compute a limit and it fails:

octave:8> intmin ('int64') - 1
I am in INT_DOUBLE_BINOP_DECL (-, int64)
x = -9223372036854775808
(-y) = -1
MIN THRESH: -9.22337e+18
MAX THRESH: 9.22337e+18
ans = 9223372036854775807
octave:9> x = intmin('int64')
x = -9223372036854775808
octave:10> --x
ans = 9223372036854775807


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]