[Octave-bug-tracker] [bug #35827] The hygepdf and hygecdf functions are numerically unstable for large populations

[Marco Caliari]

Follow-up Comment #2, bug #35827 (project octave):

Rik,

the correct answer is 7.60282189600926e-03 with Octave too.


pdf(k) = exp(gammaln(m+1)-gammaln(m-x(k)+1)-gammaln(x(k)+1)+...
gammaln(t-m+1)-gammaln(t-m-n+x(k)+1)-gammaln(n-x(k)+1)-...
gammaln(t+1)+gammaln(t-n+1)+gammaln(n+1));


but is it safe to always use this expression? I think so, the difference
should be O(eps).

Marco

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?35827>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/