[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results

[Dan Sebald]

Follow-up Comment #11, bug #54567 (project octave):

KT, the following seems to work for int64:


octave:234> dim = 2;
octave:235> xs64 = [1 6 11 16; -20 -5 14 19; intmin('int64') intmin('int64')+5
intmax('int64')-5 intmax('int64')]
xs64 =

                     1                     6                    11            
       16
                   -20                    -5                    14            
       19
  -9223372036854775808  -9223372036854775803   9223372036854775802  
9223372036854775807

octave:236> f = (sum (sign(x), dim) != 0)
f =

  1
  0
  0

octave:237> f .* x(:,1) + (x(:,2) + !f .* x(:,1) - f .* x(:,1)) / 2
ans =

   9
   5
  -1



Notice there is one small issue with the fact I had to use the colon indexing,
meaning that I'm assuming dim==2.  That means you'll have to have an if-else
statement based on "dim" with slightly different formulas for dim=1 versus
dim=2.  That's unfortunate.  I wonder if we should devise an indexing type of
function that essentially extracts a sub-matrix from a matrix the way so many
functions can act on "dim", e.g., nth_element(A,k,dim), sum(A,dim),
mean(dim).

Also, if this approach proves more efficient than all the min/maxing of the
uint64 formula in Comment #10, perhaps use of logical arrays could be used for
unit64 as well.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54567>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/