[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Kolmogorov-Smirnov test
From: |
Hamish Allan |
Subject: |
Kolmogorov-Smirnov test |
Date: |
Thu, 17 Nov 2005 17:05:05 +0000 |
Hi,
I've recently been trying to use kolmogorov_smirnov_test_2() to
compare two distributions. In order to get a feel for it, I thought
I'd try comparing two normal distributions:
octave:1> a = randn(2000,1);
octave:2> b = randn(2000,1);
octave:3> p = kolmogorov_smirnov_test_2(a,b)
p = 0.90224
octave:4> b = randn(2000,1);
octave:5> p = kolmogorov_smirnov_test_2(a,b)
warning: in /opt/local/share/octave/2.1.71/m/statistics/tests/
kolmogorov_smirnov_test_2.m near line 80, column 5:
warning: cannot compute correct p-values with ties
p = 0.98453
[These numbers look okay to me, though the warning I don't understand
(is the K-S test not used to compare e.g., distributions of school
grades, A through F, for which there would be many ties?). But that's
not my biggest problem:]
octave:6> b = randn(2000,1);
octave:7> p = kolmogorov_smirnov_test_2(a,b)
p = 0.50849
[That's a pretty low p-value! Is the K-S test actually going to be
useful if it can be so wrong about the normally-distributed datasets
as large as 2,000 points? Let's try bumping it up to 20,000...]
octave:8> a = randn(20000,1);
octave:9> b = randn(20000,1);
octave:10> p = kolmogorov_smirnov_test_2(a,b)
warning: in /opt/local/share/octave/2.1.71/m/statistics/tests/
kolmogorov_smirnov_test_2.m near line 80, column 5:
warning: cannot compute correct p-values with ties
p = 0.10441
[Huh?! One more time for luck...]
octave:11> b = randn(20000,1);
octave:12> p = kolmogorov_smirnov_test_2(a,b)
warning: in /opt/local/share/octave/2.1.71/m/statistics/tests/
kolmogorov_smirnov_test_2.m near line 80, column 5:
warning: cannot compute correct p-values with ties
p = 0.83675
[Nought point one versus nought point eight? I give up!]
So, is the K-S test really this weak, or is there a bug in the Octave
implementation?
Thanks,
Hamish
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------
- Kolmogorov-Smirnov test,
Hamish Allan <=