bug-gawk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug-gawk] MPFR rounding issue


From: Andrew J. Schorr
Subject: Re: [bug-gawk] MPFR rounding issue
Date: Mon, 20 Nov 2017 13:00:05 -0500
User-agent: Mutt/1.5.21 (2010-09-15)

On Mon, Nov 20, 2017 at 10:52:25AM -0700, david kerns wrote:
> I'm sorry I can't assist with actual test data, but compare the results of
> 2.21 vs (3.21 or 2.31)

bash-4.2$ printf "2.21\n3.21\n2.31\n" | gawk -M -v ROUNDMODE=A '{printf 
"%.1f\n", $1}'
2.3
3.3
2.4

It rounds up in all cases. Isn't that the intended behavior of
the MPFR_RNDA? The mpfr docs define it as:
      * `MPFR_RNDA': round away from zero.

> reference https://en.wikipedia.org/wiki/Rounding?#Round_half_to_even ...

But this is not "half-to-even" rounding mode. It is "round away from zero".

> BTW I might be completely off kilter on this :P
> but as Andy notes, it's just using the value the MPFR library returned

I'm not an expert in this stuff, but the behavior seems consistent with
the MPFR docs.

Maybe the problem is in the gawk docs. In the "Setting the Rounding Mode"
section of the manual, it says:

https://www.gnu.org/software/gawk/manual/html_node/Setting-the-rounding-mode.html

Rounding mode   IEEE name       ROUNDMODE
Round to nearest, ties to even  roundTiesToEven "N" or "n"
Round toward positive infinity  roundTowardPositive     "U" or "u"
Round toward negative infinity  roundTowardNegative     "D" or "d"
Round toward zero       roundTowardZero "Z" or "z"
Round to nearest, ties away from zero   roundTiesToAway "A" or "a"

So maybe that's the confusion. The gawk documentation of "A" says
"Round to nearest, ties away from zero", but the actual implementation uses
MPFR_RNDA, which rounds away from zero.

Perhaps this is simply a bug in the gawk documentation?

Regards,
Andy



reply via email to

[Prev in Thread] Current Thread [Next in Thread]