[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results

From: Dan Sebald
Subject: [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results
Date: Sun, 26 Aug 2018 03:29:38 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Follow-up Comment #12, bug #54567 (project octave):

BTW, we should at BIST tests for all these cases.

Also, we know mean() probably has the issue we are trying to avoid:

octave:253> mean([intmin('int64')+5 intmax('int64')-5],'native')
ans = 0
octave:254> mean([int64(-1) int64(0)],'native')
ans = -1

I.e., the large magnitude int64 numbers have a loss of precision in mean() so
the result doesn't match the formula of Comment #11.


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]