[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in n

From: Dan Sebald
Subject: [Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction
Date: Tue, 28 Aug 2018 03:25:54 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Follow-up Comment #9, bug #54572 (project octave):

octave:4> __octave_config_info__ ("CC")
ans = gcc
octave:5> exit
@linux ~/ $ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609

I wonder if this is just that floating point 53-bit mantissa (significand)
issue when converting to int64:

octave:14> int64(-9223372036854775808)
ans = -9223372036854775808
octave:15> int64(-9223372036854775807)
ans = -9223372036854775808
octave:16> int64(-9223372036854775806)
ans = -9223372036854775808

Then again, maybe not:

octave:45> x = intmin("int64")
x = -9223372036854775808
octave:46> --x
ans = 9223372036854775807
octave:47> class(x)
ans = int64


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]