PSPP-BUG: [bug #51235] Wrong rounding of decimals in FACTOR output?

[Jaap Verhage]

URL:
  <http://savannah.gnu.org/bugs/?51235>

                 Summary: Wrong rounding of decimals in FACTOR output?
                 Project: PSPP
            Submitted by: jaapv
            Submitted on: Tue 13 Jun 2017 07:57:26 PM UTC
                Category: Numerical Errors
                Severity: 5 - Average
                  Status: None
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None
                  Effort: 0.00

    _______________________________________________________

Details:

In the "Communalities table" in the output of a FACTOR command, PSPP reports a
value of .93. When instructed to display three decimals via the SET FORMAT
command and repeating the FACTOR command, this displays as .935. Shoudn't that
be .94 instead of .93 with two decimals? (It's .935000 when instructed to
display six decimals.)

Regards, Jaap.





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?51235>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/