[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #36133] num2str displays more than 16 signific

From: Rik
Subject: [Octave-bug-tracker] [bug #36133] num2str displays more than 16 significant digits for large integer inputs
Date: Wed, 25 Apr 2018 12:58:54 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0

Follow-up Comment #8, bug #36133 (project octave):

Adding in jwe because the behavior is fascinating.  From Hartmut's tests in
comment #3, the Matlab sprintf function returns

sprintf ('%30.0f', 1e25)
ans =

where the '1' in the sectond position is very real.

Octave's answer for the same code is

ans =     10000000000000000905969664

If I align the two

    10000000000000001000000000     # Matlab
    10000000000000000905969664     # Octave

it seems pretty clear what Matlab has done.  First, Octave uses the C library
routine for sprintf so we get whatever default behavior is implemented when
the precision exceeds that of the underlying number.  But Matlab has written a
shim layer for sprintf which calls the sprintf library, but then cleans up the
result.  However, they haven't been too clever about it.  IEEE-854 doubles
have a variable precision of 15-17 decimal digits.  The extra '1' occurs
because they are rounding the digit '9' which is the 18 significant figure and
not valid.

In any case, once Matlab has printed 17 digits, it pads the rest of the string
with zeros.  This is pretty easy to do in C++ with the substr() function or
operator[] access and a for loop.

So should Octave do something similar?  And if so, where should it be done? 
It could be done just in our sprintf routine, but then other actions like
printf would diverge in behavior.  That means it would probably be better to
do it in octave::stream.


Reply to this item at:


  Message sent via/by Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]