[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54414] Not recognizing that indices greater t
|
From: |
Rik |
|
Subject: |
[Octave-bug-tracker] [bug #54414] Not recognizing that indices greater than (roughly) 2^63 or 20 digits are too large |
|
Date: |
Wed, 1 Aug 2018 15:23:51 -0400 (EDT) |
|
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0 |
Follow-up Comment #6, bug #54414 (project octave):
But isn't this just bug #45945 again? In Octave there is no way to enter the
number in the interpreter and not have it lose precision with respect to
(u)int64 because the interpreter always reads doubles.
x = 9223372036854775807
x = 9.2234e+18
sprintf ("%20f\n", x)
ans = 9223372036854775808.000000
The number has already lost one digit of precision. Now find out what eps
is.
eps (x)
ans = 2048
And if I try and read a number that is distinguishable from the max
octave_idx_type then I do get an error.
fread (fid, [x+eps(x), x+eps(x)], 'single')
error: fread: dimension too large for Octave's index type
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54414>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/