[Octave-bug-tracker] [bug #32991] format long shows more decimal digits than can possibly be accurate

[anonymous]

Follow-up Comment #3, bug #32991 (project octave):

Consider the case of "format long eng" which is even worse. In engineering
format (exponent divisble by 3) we have three choices

1) Displaying exactly the right number of digits but with a ragged display of
the numbers which is rather ugly
2) Possibly display too few digits, but get the alignment of the number in a
matrix correct
3) Possibly display too many digits, but get the alignment of the number in
the matrix correct

consider

format long eng
1000*randn(10,3)

in Octave and Matlab. Matlab made the choice 1) and in Octave I made the
choice 3).. 

If this bug report is considered a bug then the behavior of "format long eng"
is also a bug. Frankly I consider neither this bug report nor the behavior of
"format long eng" as a bug. Users of floating point arithmetic should know the
limitations or they shouldn't be using floating point arithmetic at all.

D (David Bateman posted anonymously as I don't have access to my dev machine)
 

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?32991>

_______________________________________________
  Message posté via/par Savannah
  http://savannah.gnu.org/