octave-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #53167] format long doesn't adjust precision t


From: Rik
Subject: [Octave-bug-tracker] [bug #53167] format long doesn't adjust precision to 7 digits when displaying single() values
Date: Mon, 19 Feb 2018 19:46:16 -0500 (EST)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0

Follow-up Comment #8, bug #53167 (project octave):

I don't think digits10 does the right thing.  I tried


   cout << "Max sig. digits double: " << numeric_limits<double>::digits10 <<
endl;
   cout << "Max sig. digits single: " << numeric_limits<float>::digits10 <<
endl;


and get


Max sig. digits double: 15
Max sig. digits single: 6




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53167>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




reply via email to

[Prev in Thread] Current Thread [Next in Thread]