[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #53167] format long doesn't adjust precision t

From: John W. Eaton
Subject: [Octave-bug-tracker] [bug #53167] format long doesn't adjust precision to 7 digits when displaying single() values
Date: Sun, 18 Feb 2018 12:47:35 -0500 (EST)
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0

Follow-up Comment #3, bug #53167 (project octave):

Currently, a side effect of "format long" and "format short" is to set
output_precision.  But that's not specific to single or double values.  So we
have a choice or two to make.

Should we introduce a separate output_precision variable for single precision
values and have format long/short set both of those values?

If not, then should we limit the display of values to the maximum number of
digits that double and single values can represent
(std::numeric_limits<T>::digits10 in C++)?

If we have a limit, should that limit apply to both single and double
precision values, and should it be imposed on the value of  the
output_precision variable(s) or just as a limit inside the print functions?

Should we eliminate the output_precision and format_max_field_width variables
and just use the format function to set long/short/whatever format styles?

None of these changes would affect *printf, so you could always do something
like "fprintf ('%.200f\n', val)" if you want to see the digits that the
floating point conversion routine generates when you ask for more precision
than is actually available.


Reply to this item at:


  Message sent via/by Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]