pspp-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

t-test gets NaN instead of 0 for significance on x86-64


From: Ben Pfaff
Subject: t-test gets NaN instead of 0 for significance on x86-64
Date: Tue, 01 Sep 2009 21:15:49 -0700
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux)

Some time ago, Matej Cepl <address@hidden> reported that
t-test-alpha3.sh in 0.6.2-pre5 failed on x86-64 with GCC 4.4:

> PASS: tests/bugs/t-test-alpha3.sh
> 19c19
> < #Pair 0|A & B#3|      1.000| NaN#
> ---
>> #Pair 0|A & B#3|      1.000|.000#
> compare output
> FAILED
> FAIL: tests/bugs/t-test-paired.sh

This evening, I've run the same test on an x86-64 machine
(bellini.debian.org) with GCC 4.3.2 and I reproduce this test
failure.  I also get the same failure with 0.6.1, although I have
to run the test by hand there since this test was new in
0.6.2-pre5.

After some fussing, I tracked the source of the NaN to this
calculation in pscbox() in src/language/stats/t-test.q:

      double correlation_t =
        pairs[i].correlation * sqrt (df) /
        sqrt (1 - pow2 (pairs[i].correlation));

In this particular test case, pairs[i].correlation is almost
exactly 1.0, such that 1 - pow2 (pairs[i].correlation) comes out
just slightly negative, making the square root yield NaN.

John, do you have a suggestion for the correct fix?  I don't know
enough about the math here to say.
-- 
Ben Pfaff 
http://benpfaff.org




reply via email to

[Prev in Thread] Current Thread [Next in Thread]