[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in n
From: |
Dan Sebald |
Subject: |
[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction |
Date: |
Tue, 28 Aug 2018 16:15:17 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0 |
Follow-up Comment #15, bug #54572 (project octave):
Oh, I see. Yeah, I suspect it is that float-53/int-64 issue again. Recall,
before that issue manifested as a float index mapped to an integer negative
number (or something like that). Here, I would guess that the arithmetic
with
// Compute proper thresholds.
static const S thmin = compute_threshold (static_cast<S> (min_val ()),
min_val ());
static const S thmax = compute_threshold (static_cast<S> (max_val ()),
max_val ());
[snip]
compute_threshold (S val, T orig_val)
{
val = std::round (val); // Fool optimizations (maybe redundant)
// If val is even, but orig_val is odd, we're one unit off.
if (orig_val % 2 && val / 2 == std::round (val / 2))
// FIXME: is this always correct?
val *= (static_cast<S> (1) - (std::numeric_limits<S>::epsilon () / 2));
return val;
}
is producing a threshold "thmin" that is one resolution bit on the negative
side of LINT_MIN or std::numeric_limits<int64_t>::min. No time to investigate
right now.
However, I'm not certain of the above guess because the decrement operator
doesn't appear to compute a limit and it fails:
octave:8> intmin ('int64') - 1
I am in INT_DOUBLE_BINOP_DECL (-, int64)
x = -9223372036854775808
(-y) = -1
MIN THRESH: -9.22337e+18
MAX THRESH: 9.22337e+18
ans = 9223372036854775807
octave:9> x = intmin('int64')
x = -9223372036854775808
octave:10> --x
ans = 9223372036854775807
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54572>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, (continued)
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dmitri A. Sergatskov, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dmitri A. Sergatskov, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Rik, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dmitri A. Sergatskov, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, John W. Eaton, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Rik, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction,
Dan Sebald <=
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, John W. Eaton, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Rik, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, John W. Eaton, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Rik, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Dan Sebald, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, Rik, 2018/08/28
- [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction, John W. Eaton, 2018/08/28