[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in n

From: Dan Sebald
Subject: [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction
Date: Wed, 29 Aug 2018 18:19:31 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Follow-up Comment #32, bug #54572 (project octave):

@JWE: You modified both the add() and sub() routines to use sign comparison
rather than adding a value.  It looks to me that the purpose of that sign test
code is exactly the same in both cases.  So rather than this (the

  // This is very similar to addition.
  static T
  sub (T x, T y)
    // The typecasts do nothing, but they are here to prevent an optimizing
    // compiler from interfering.  Also, the signed operations on small types
    // actually return int.
    T u = static_cast<UT> (x) - static_cast<UT> (y);
    T ux = u ^ x;
    T uy = u ^ ~y;
    if ((ux & uy) < 0)
        u = (__signbit (~u)
             ? octave_int_base<T>::min_val ()
             : octave_int_base<T>::max_val ());

which uses the 1's complement, how about making them consistent?  I.e.,
__signbit(~u) = !__signbit(u) = !(u < 0), so switch around the ternary
operator arguments and get

        u = (u < 0
             ? octave_int_base<T>::max_val ()
             : octave_int_base<T>::min_val ());

the same as for add().  In some sense, that last portion of the routine is
independent of the modulo arithmetic that took place before it.


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]