output rounding bug in vasnprintf %e format implementation

[Ben Pfaff]

[This issue arose in examination of bug 22924 against GNU PSPP,
which may be viewed at https://savannah.gnu.org/bugs/?22924.]

The %e format implemented by gnulib's vasnprintf (when
NEED_PRINTF_DOUBLE is defined) does not round properly in all
cases.  Here is one example: 999.95 printed with %.4e should
yield 9.9995e+02, but actually yields 1.0000e+03 with the gnulib
implementation.  I'm appending to this email a patch to the
printf-posix and fprintf-posix test cases that finds this
problem.

I assume that there is a similar bug with %Le, but I have not
checked.

The problem appears to be around line 3180 of vasnprintf.c.  This
code calls floorlog10 to estimate the required exponent, which
result can be off by 1, and then scale10_round_decimal_double to
render the value with the selected exponent.  By adding a printf
around here, I can see that floorlog10 chooses an exponent of 3.
Of course then scale10_round_decimal_double renders that, quite
reasonably, as 1.0000, and vasnprintf accepts that as correct.

A solution might be to make floorlog10 precise.  This might
efficiently be possible by, for example, maintaining a table of
powers of 10 generated at compile time or at program startup and
performing binary search within it.

Bruno, do you have an idea for a fix?

Thanks,

Ben.

diff --git a/tests/test-fprintf-posix.h b/tests/test-fprintf-posix.h
index fc44351..239d12a 100644
--- a/tests/test-fprintf-posix.h
+++ b/tests/test-fprintf-posix.h
@@ -121,6 +121,9 @@ test_function (int (*my_fprintf) (FILE *, const char *, 
...))
   /* Precision.  */
   my_fprintf (stdout, "%.LF %d\n", 1234.0L, 33, 44, 55);
 
+  /* Rounding. */
+  my_fprintf (stdout, "%.2f %.4e %d\n", 999.95, 999.95, 33, 44, 55);
+
   /* Test the support of the POSIX/XSI format strings with positions.  */
 
   my_fprintf (stdout, "%2$d %1$d\n", 33, 55);
diff --git a/tests/test-printf-posix.h b/tests/test-printf-posix.h
index 37caf86..62992c2 100644
--- a/tests/test-printf-posix.h
+++ b/tests/test-printf-posix.h
@@ -123,6 +123,9 @@ test_function (int (*my_printf) (const char *, ...))
   /* Precision.  */
   my_printf ("%.LF %d\n", 1234.0L, 33, 44, 55);
 
+  /* Rounding. */
+  my_printf ("%.2f %.4e %d\n", 999.95, 999.95, 33, 44, 55);
+
   /* Test the support of the POSIX/XSI format strings with positions.  */
 
   my_printf ("%2$d %1$d\n", 33, 55);
diff --git a/tests/test-printf-posix.output b/tests/test-printf-posix.output
index 4a6a170..7996622 100644
--- a/tests/test-printf-posix.output
+++ b/tests/test-printf-posix.output
@@ -29,4 +29,5 @@ inf 33
 0.000000 33
 00001234.000000 33
 1234 33
+999.95 9.9995e+02 33
 55 33

-- 
Ben Pfaff 
http://benpfaff.org