axiom-developer
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Axiom-developer] i-output and decimal width


From: Stephen Wilson
Subject: [Axiom-developer] i-output and decimal width
Date: 03 Aug 2007 15:03:49 -0400
User-agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.4

*,

There was some discussion a while ago about the rounding behaviours of
GCL w.r.t base 10 logarithms, and the impact it has in counting the
number of digits in a radix 10 expansion.

We have the following code in i-output.boot.pamphlet:

   u < MOST_-POSITIVE_-LONG_-FLOAT => 1+negative+FLOOR ((LOG10 u) + 0.0000001)
   -- Rough guess: integer-length returns log2 rounded up, so divide it by
   -- roughly log2(10). This should return an over-estimate, but for objects
   -- this big does it matter?
   FLOOR(INTEGER_-LENGTH(u)/3.3)


Here, u is a positive integer.  The first case which tests against a
bound on most-positive-long-float is almost OK but it adjusts the call
to LOG10 by a small delta which is not enough on some machines.

The final case using integer-length is just not right.  The algorithm
returns an extra `digit' to account for a minus sign, which is missed
by this calculation, and it also returns an over estimation.


Could I suggest the following to replace these two cases:

    (+ 1 |negative| (round (log n 10)))






reply via email to

[Prev in Thread] Current Thread [Next in Thread]