octave-bug-tracker
[Top][All Lists]

## [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results

 From: Dan Sebald Subject: [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results Date: Sun, 26 Aug 2018 03:25:46 -0400 (EDT) User-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

```Follow-up Comment #11, bug #54567 (project octave):

KT, the following seems to work for int64:

octave:234> dim = 2;
octave:235> xs64 = [1 6 11 16; -20 -5 14 19; intmin('int64') intmin('int64')+5
intmax('int64')-5 intmax('int64')]
xs64 =

1                     6                    11
16
-20                    -5                    14
19
-9223372036854775808  -9223372036854775803   9223372036854775802
9223372036854775807

octave:236> f = (sum (sign(x), dim) != 0)
f =

1
0
0

octave:237> f .* x(:,1) + (x(:,2) + !f .* x(:,1) - f .* x(:,1)) / 2
ans =

9
5
-1

Notice there is one small issue with the fact I had to use the colon indexing,
meaning that I'm assuming dim==2.  That means you'll have to have an if-else
statement based on "dim" with slightly different formulas for dim=1 versus
dim=2.  That's unfortunate.  I wonder if we should devise an indexing type of
function that essentially extracts a sub-matrix from a matrix the way so many
functions can act on "dim", e.g., nth_element(A,k,dim), sum(A,dim),
mean(dim).

Also, if this approach proves more efficient than all the min/maxing of the
uint64 formula in Comment #10, perhaps use of logical arrays could be used for
unit64 as well.

_______________________________________________________

<http://savannah.gnu.org/bugs/?54567>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/

```