[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #36133] num2str displays more than 16 signific

From: Rik
Subject: [Octave-bug-tracker] [bug #36133] num2str displays more than 16 significant digits for large integer inputs
Date: Wed, 25 Apr 2018 17:43:54 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0

Follow-up Comment #10, bug #36133 (project octave):

Results from Matlab will be interesting, but I doubt that they test that each
number to be printed is an exact power of two.  There are 1024 of those
numbers compared to 2^53 (9e15) numbers representable in the mantissa.  It
really, really fails the 80/20 rule to worry about them.

According to Matlab's documentation for both sprintf and num2str, they are
aware of the issue


If you specify a precision operator for floating-point values that exceeds the
precision of the input numeric data type, the results might not match the
input values to the precision you specified. The result depends on your
computer hardware and operating system.

Also, according to the documentation for num2str

s = num2str(A) converts a numeric array into a character array that represents
the numbers. The output format depends on the magnitudes of the original

That's fine, but my guess is that Matlab just uses a form of the %g format for
both floating point and integer numbers.  Octave, however, tries to switch
format based on whether the input is an integer.  The code is

if (any (x(valid) != fix (x(valid))))
  ## Floating point input
  ndgt = max (ndgt + 5, 5);   # Keep at least 5 significant digits
  ndgt = min (ndgt, 16);      # Cap significant digits at 16
  fmt = sprintf ("%%%d.%dg", ndgt+7, ndgt);
  ## Integer input
  ndgt += 3;
  if (any (! valid))
    ndgt = max (ndgt, 5);     # Allow space for Inf/NaN
  ## FIXME: Integers must be masked to show only 16 significant digits
  ##        See test case for bug #36133 below
  fmt = sprintf ("%%%d.0f", ndgt);

I bet changing the integer case would fix some of this. 


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]