Re: [bug-gawk] MPFR rounding issue

[david kerns]

I'm sorry I can't assist with actual test data, but compare the results of 2.21 vs (3.21 or 2.31)

reference https://en.wikipedia.org/wiki/Rounding?#Round_half_to_even ... BTW I might be completely off kilter on this :P

but as Andy notes, it's just using the value the MPFR library returned

On Mon, Nov 20, 2017 at 9:28 AM, Andrew J. Schorr <address@hidden> wrote:

Hi,

So this one-liner shows the issue, I think:
   gawk -M 'BEGIN {ROUNDMODE = "A"; printf "%.1f\n", 2.21}'
The output is 2.3

It renders this with a call to mpfr_snprintf with a format argument
of "%*.*R*f", and then passes 0 as the width and 1 as the precision.

So why is this wrong? The comment in the discussion says:
   So perhaps the Gawk maintainers are misunderstanding how MPFR_RNDA works in MPFR.
How so? I think we're simply printing the result from mpfr_snprintf.
Is that inadvisable? What are we misunderstanding?

Maybe I really don't understand this stuff, but when MPFR rounds 2.21
to 1 digit of precision, it has a choice of 2.2 or 2.3, and it picks the
one with greater magnitude because that is the requested rounding mode.
What else should it do?

Regards,
Andy

On Mon, Nov 20, 2017 at 04:01:22PM +0100, David Kaspar [Dee'Kej] wrote:
> Ah, sorry for the missing Subject... :( Somehow Google didn't stop me this
> time before sending it... :-/
>
> David Kaspar [Dee'Kej]
> *Associate Software Engineer*
> *Brno, Czech Republic*
>
> RED HAT | TRIED. TESTED. TRUSTED.
> Every airline in the Fortune 500 relies on Red Hat.
> Find out why at Trusted | Red Hat <http://www.redhat.com/en/about/trusted>.
>
> On Mon, Nov 20, 2017 at 3:52 PM, David Kaspar [Dee'Kej] <address@hidden>
> wrote:
>
> > Hello folks,
> >
> > we have received a new BZ report for a different behaviour of Gawk when
> > MPFR is used and when it is not. Reproducer script:
> >
> > BEGIN {
> >   ROUNDMODE = "A"
> >   x = 2.21
> >   printf "%4.2f\n",x
> >   printf "%3.1f\n",x
> >   exit
> > }
> >
> > $ gawk -f ./script
> > 2.21
> > 2.2
> >
> > $ gawk -M -f ./script
> > 2.21
> > 2.3
> >
> > * https://bugzilla.redhat.com/show_bug.cgi?id=1514953
> >
> > The interesting discussion about this was also here -
> > https://forums.fedoraforum.org/showthread.php?316207-
> > Setting-gawk-ROUNDMODE&p=1798362#post1798362 :
> > "Hmm, looks like Gawk is just (unintentionally) doing with ROUNDMODE="A"
> > what MPFR is doing with its MPFR_RNDA: it's rounding away from zero at all
> > times, not just for ties."
> >
> > Is this really a bug, or just user expecting something wrong from gawk?
> >
> > Best regards,
> >
> > David Kaspar [Dee'Kej]
> > *Associate Software Engineer*
> > *Brno, Czech Republic*
> >
> > RED HAT | TRIED. TESTED. TRUSTED.
> > Every airline in the Fortune 500 relies on Red Hat.
> > Find out why at Trusted | Red Hat <http://www.redhat.com/en/about/trusted>
> > .
> >

--
Andrew Schorr                      e-mail: address@hiddencom
Telemetry Investments, L.L.C.      phone:  917-305-1748
545 Fifth Ave, Suite 1108          fax:    212-425-5550
New York, NY 10017-3630