[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in n

From: Dan Sebald
Subject: [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction
Date: Tue, 28 Aug 2018 21:43:53 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Follow-up Comment #26, bug #54572 (project octave):


Here's a little patch to proof-of-concept the use of the GNU-lib builtins.  It
only works for int64 because I hard-coded the use of
__builtin_saddll_overflow(), but if you could figure out a way to use the
template T to substitute the proper builtin routine (conditioned on the GCC
compiler being used), there couldn't be any faster routine.

It works, and I think I have the < 0 (versus <= 0) correct, but there should
be plenty of corner cases added to the BIST:

octave:1> intmin ('int64')
ans = -9223372036854775808
octave:2> intmin ('int64') - 1
ans = -9223372036854775808
octave:3> intmin ('int64') + 1
ans = -9223372036854775807
octave:4> intmin ('int64') + intmin ('int64')
ans = -9223372036854775808
octave:5> intmin ('int64') - intmin ('int64')
ans = 0
octave:6> intmin ('int64') - intmax ('int64')
ans = -9223372036854775808
octave:7> intmax ('int64') + intmax ('int64')
ans = 9223372036854775807
octave:8> intmax ('int64') - intmin ('int64')
ans = 9223372036854775807

BTW, something else that would be nice is a special type/class variable
definition that exists only in that intermediate world of the interpreter;
call it a "long_double".  Rather than ASCII numbers being immediately
converted to double they could have the significand stored as a long 64 bit
while the exponent is stored in a double as 1eX.  So if the user types

octave:7> int64(-9223372036854775806)
ans = -9223372036854775808

the number inside the function argument list would be treated as

(long_double) -9223372036854775806 x 1e0

and with the proper construct the int64() routine could convert the
long_double back to a long without loss of resolution.  It would only be when
an assignment is made that the double_long gets converted to double.  (I.e.,
the user would never be able to create or see any type of class called

(file #44893)

Additional Item Attachment:

File name: int64_overflow_proof_of_concept.diff Size:1 KB


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]