octave-bug-tracker
[Top][All Lists]

## [Octave-bug-tracker] [bug #53166] format long displays 15 significant di

 From: Rik Subject: [Octave-bug-tracker] [bug #53166] format long displays 15 significant digits, rather than 15 digits after decimal point (Matlab Compatibility) Date: Fri, 16 Feb 2018 12:02:03 -0500 (EST) User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0

```Follow-up Comment #5, bug #53166 (project octave):

I asked for a test of Matlab on the Maintainer's list:

Matlab:

format long
x = pi;
y = [x; x*10; x*100; x*1000]
single (y)

y =

1.0e+03 *

0.003141592653590
0.031415926535898
0.314159265358979
3.141592653589793

ans =

4×1 single column vector

1.0e+03 *

0.0031416
0.0314159
0.3141593
3.1415928

In Octave, for the last value

format long
fixed_point_format (true)
x = pi;
y = [x; x*10; x*100; x*1000]
y =

1.0e+03 *

0.003141592653590
0.031415926535898
0.314159265358979
3.141592653589793

When configured with exactly the same options, Matlab and Octave are
equivalent.  If I disable fixed_point_format the results are

fixed_point_format (false)
y
y =

3.14159265358979
31.41592653589793
314.15926535897933
3141.59265358979292

The Matlab equivalent display for pi has one extra digit on it as mentioned
earlier:

Matlab

format long
3.141592653589793

In general, the IEEE-854 double precision format has 15-17 significant digits
(https://en.wikipedia.org/wiki/Double-precision_floating-point_format).  A
reasonable argument can be made that Octave should be showing the "average"
case of 16 significant digits (rather than 15 which is what we do now).  This
would imply that we increase by 1 the number of digits we show in the "format
long" case.

_______________________________________________________

<http://savannah.gnu.org/bugs/?53166>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

```