|
From: | Mike Miller |
Subject: | [Octave-bug-tracker] [bug #53167] format long doesn't adjust precision to 7 digits when displaying single() values |
Date: | Mon, 19 Feb 2018 21:21:15 -0500 (EST) |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0 |
Follow-up Comment #9, bug #53167 (project octave): It's doing the right thing. But we might want to use digits10 + 1 for floating point types. The value of digits10 isn't the maximum number of digits that can be represented at all, it's the maximum number of digits that can always be represented without any loss of precision. flintmax("double") is close to 1e16, but not quite. and flintmax("single") is closer to 1e7. So 15 and 6 are correct. http://en.cppreference.com/w/cpp/types/numeric_limits/digits10 _______________________________________________________ Reply to this item at: <http://savannah.gnu.org/bugs/?53167> _______________________________________________ Message sent via/by Savannah http://savannah.gnu.org/
[Prev in Thread] | Current Thread | [Next in Thread] |