[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in n

From: John W. Eaton
Subject: [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction
Date: Tue, 28 Aug 2018 16:56:50 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0

Follow-up Comment #20, bug #54572 (project octave):

Ah, instead of

      u = octave_int_base<T>::max_val () + __signbit (~u);

I think we can write:

      u = (__signbit (~u)
           ? std::numeric_limits<T>::min ()
           : std::numeric_limits<T>::max ());

because it seems as though the intent was to either set the result to the max
value for the type (signbit is 0) or, if signbit is 1, add 1 to the max value
and wrap around to the min value.  Why not set the min and max directly,
simply by checking the sign?

Also, I think octave_int_base<T>::max_val () was the wrong thing here anyway,
because if I'm reading the code correctly, that returns an octave_int<T> type,
so wouldn't the operator + applied to that value send us back through all the
octave_int code for adding the signbit (0 or 1) value??

The original code seems like it tried to be a bit too tricky for simply
setting the max or min value if an overflow condition is detected.

I'm hereby coining a new term that may be used to describe this type of
problem in Octave:  Jaroslavimization.  :-)


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]