[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: rounding bug (?) in gawk 3.1.2

From: Stepan Kasal
Subject: Re: rounding bug (?) in gawk 3.1.2
Date: Sun, 22 Feb 2004 11:37:20 +0100
User-agent: Mutt/1.4.1i


On Fri, Feb 20, 2004 at 03:10:20PM +0100, luca de filippis wrote:
> awk 'BEGIN { for (i=0; i<=9; i++)
>      printf "%2i %5.2f \n", i, 0.01+i/1000 }'
>  5  0.01

this is not a bug, it's a consequence of using float arithmetics.
Similar things happen quite often, for example:

$ gawk 'BEGIN{printf("%.1f %.1f\n",.35,.45)}'
0.3 0.5

the reason is that gawk uses double as the internal numeric
representation.  As this data type has only a finite precision,
one can never tell which binary number will be chosen to represent
the decimal number.

And if the actual internal representation of .35 is smaller than .35,
even by an almost infinite difference, then the rouding is screwed up.

When you work with floating arithmetics, you can never rely that
2 * (x / 2) will be again x.

As these problems can generally be called "consequences" of double-
precision arithmetics, they are not mentioned in the BUGS section
of the manpage.

Hope this explains it,
        Stepan Kasal

reply via email to

[Prev in Thread] Current Thread [Next in Thread]