[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results
From: |
Dan Sebald |
Subject: |
[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results |
Date: |
Sun, 26 Aug 2018 03:29:38 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0 |
Follow-up Comment #12, bug #54567 (project octave):
BTW, we should at BIST tests for all these cases.
Also, we know mean() probably has the issue we are trying to avoid:
octave:253> mean([intmin('int64')+5 intmax('int64')-5],'native')
ans = 0
octave:254> mean([int64(-1) int64(0)],'native')
ans = -1
I.e., the large magnitude int64 numbers have a loss of precision in mean() so
the result doesn't match the formula of Comment #11.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54567>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, (continued)
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Kai Torben Ohlhus, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/24
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Kai Torben Ohlhus, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Michael Godfrey, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Rik, 2018/08/25
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results,
Dan Sebald <=
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Michael Godfrey, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Dan Sebald, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Michael Godfrey, 2018/08/26
- [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results, Rik, 2018/08/27