Re: [bug-gawk] macOS: mpfrsqrt check fails on gawk-4.2-stable and master branches

[Andrew J. Schorr]

On Wed, Feb 07, 2018 at 09:00:30PM +0100, Hermann Peifer wrote:
> I couldn't find anything wrong with the extensions on my side. So I
> patched mpfrsqrt.awk and ended up with the attached log file which is
> completely different from yours, with only 9 iterations rather than 11.

Thanks. We are making progress. So the mismatch starts in the first
calculation.

On linux, we have:
123456790123456790123456790123456790123456790123456790123454320987654320987654320987654320987654320987654320987654321/1234567901234567901234567901234567901234567901234567901234
 = 100000000000000000000000000000000000000000000000000000000044

Whereas on MacOS:

123456790123456790123456790123456790123456790123456790123454320987654320987654320987654320987654320987654320987654321/1234567901234567901234567901234567901234567901234567901234
 = 99999999999999997168788049560464200849936328366177157906432

This looks like a precision problem. On Linux:

bash-4.2$ echo 
123456790123456790123456790123456790123456790123456790123454320987654320987654320987654320987654320987654320987654321
 1234567901234567901234567901234567901234567901234567901234 | ./gawk -M 
'{printf "%d\n", $1/$2}'
99999999999999997168788049560464200849936328366177157906432

bash-4.2$ echo 
123456790123456790123456790123456790123456790123456790123454320987654320987654320987654320987654320987654320987654321
 1234567901234567901234567901234567901234567901234567901234 | ./gawk -M -v 
PREC=400 '{printf "%d\n", $1/$2}'
100000000000000000000000000000000000000000000000000000000044

What do you see on MacOS?

That being said, I don't know why the PREC value is set higher in
test/mpfrsqrt.awk, since we don't actually set it. I am confused...

If you patch mpfrsqrt.awk to add PREC = 400 near the top, does it give
the right answer?

Regards,
Andy