[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #49961] sign_test() returns incorrect p-value
From: |
Kirill Pushkaryov |
Subject: |
[Octave-bug-tracker] [bug #49961] sign_test() returns incorrect p-value |
Date: |
Thu, 5 Jan 2017 04:24:36 +0000 (UTC) |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0 |
Follow-up Comment #3, bug #49961 (project octave):
As far as the sign test is not a general purpose algorithm, it sounds
reasonable to leave it to the statistics package.
Unfortunately, it seems that signtest() is not entirely correct too, at least
when tail = 'right':
switch lower(tail)
case 'both'
w = min(sum(x<my),sum(x>my));
pl = binocdf(w, n, 0.5);
p = 2*min(pl,1-pl);
case 'left'
w = sum(x<my);
p = binocdf(w, n, 0.5);
case 'right'
w = sum(x>my);
p = 1 - binocdf(w, n, 0.5);
otherwise
Here we have p = 1 - binocdf(w, n, 0.5) = 1 - P(X <= w) = P(X > w), so X = w
is omitted. But p-value is by definition the probability to observe _the same_
or more extreme value under H0.
More problems I've noticed with signtest():
1) ties aren't excluded
2) incorrect results with tail = 'left':
>> [pval, h] = signtest([-ones(1, 1000) 1], 0, 'tail', 'left')
pval = 1
h = 0
We checked if a vector of a thousand -1's and one 1 has a median less than 0,
and signtest() rejected this hypothesis! In the source we have p = binocdf(w,
n, 0.5) = P(X <= sum(x < 0)). So, the more components of x are less than 0,
the closer p is to 1, but it should be the other way around.
@Kai Torben Ohlhus: Alas, I don't have any tests. I've found a paper on the
sign test (Dixon, Wilfrid J., and Alexander M. Mood. "The statistical sign
test." Journal of the American Statistical Association 41.236 (1946):
557-566), and it includes a table of critical values, but that wouldn't be
convenient for tests. I suppose, the needed values could be generated using
some trustworthy statistics software.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?49961>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/