[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug-gawk] Error Gawk Manual FP Precision

From: ricky
Subject: [bug-gawk] Error Gawk Manual FP Precision
Date: Thu, 16 Aug 2018 07:11:20 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0

Dear bug-gawk:

The section on comparing two floating point values correctly recommends making avoiding simple equality comparisons in favor of ensuring the values are close enough.

The code given in Be Careful Comparing Values is clearly wrong, though:

The general wisdom when comparing floating-point values is to see if they are within some small range of each other (called a delta, or tolerance). You have to decide how small a delta is important to you. Code to do this looks something like the following:

delta = 0.00001                 # for example
difference = abs(a) - abs(b)    # subtract the two values
if (difference < delta)
    # all ok
    # not ok


If the desired value, a is -1000, and b is +1000, the answer is off by 2000. But difference will be zero.

Thanks for all of your work,


reply via email to

[Prev in Thread] Current Thread [Next in Thread]