octave-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #53167] format long doesn't adjust precision t


From: Mike Miller
Subject: [Octave-bug-tracker] [bug #53167] format long doesn't adjust precision to 7 digits when displaying single() values
Date: Mon, 19 Feb 2018 21:21:15 -0500 (EST)
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0

Follow-up Comment #9, bug #53167 (project octave):

It's doing the right thing. But we might want to use digits10 + 1 for floating
point types. The value of digits10 isn't the maximum number of digits that can
be represented at all, it's the maximum number of digits that can always be
represented without any loss of precision.

flintmax("double") is close to 1e16, but not quite. and flintmax("single") is
closer to 1e7. So 15 and 6 are correct.

http://en.cppreference.com/w/cpp/types/numeric_limits/digits10

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53167>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




reply via email to

[Prev in Thread] Current Thread [Next in Thread]