[Octave-bug-tracker] [bug #56941] format rat or rats display 0 rather than rational approximation for small numbers

[Rik]

Follow-up Comment #6, bug #56941 (project octave):

This is going to be rather difficult to fully fix short of overhauling all of
the rational print routines.

In Matlab, they call the equivalent of rat with a specific tolerance


[N,D] = rat(X,tol), where tol is
min(10^(-(strlen-1)/2)*norm(X(isfinite(X)),1),.1)


Next they form the string "N/D" and then if that generated string is greater
than strlen they display "*".

In pr-output.cc, Octave calls out to the function rational_approx (val,
strlen) which is in liboctave/util/oct-string.cc.  This function generates a
string of no more than the specified length (strlen).  This produces strange
results because a small positive number which can just be represented in the
desired length will be returned, but a small negative number with the same
representation, is returned as '0' because the minus sign makes it one longer
than the requested strlen.

Additionaly, the calling routine in pr-output.cc then looks to see if the
returned string is greater than strlen, and if so replaces it with '*'.  But
this doesn't happen for small numbers because the earlier routine has already
clipped the result and exited when this condition is met.

I can make a few alterations to improve things, but full Matlab-compatibility
would require a big rewrite.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?56941>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/