[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results

[Kai Torben Ohlhus]

Follow-up Comment #2, bug #54567 (project octave):

Matlab R2018a returns "140" in both cases of the original comment.

The patch file #44857 of comment #1 works


-    retval = sum (nth_element (x, k:k+1, dim), dim, "native") / 2;
+    retval = mean (nth_element (x, k:k+1, dim), dim, "native");


But I think it makes things more complicated and maybe slower for large data?!
 Basically, mean() would perform the computation with double precision and
finally convert the result to the input data type.  Why not doing the very
same without additional function calls in one line and tidy up the nasty
"logical" exception:


-    retval = sum (nth_element (x, k:k+1, dim), dim, "native") / 2;
-    if (islogical (x))
-      retval = logical (retval);
-    endif
+    ## Perform the averaging in double precision to avoid intermediate
+    ## overflows of finite data types like "int8".
+    retval = cast (sum (nth_element (x, k:k+1, dim), dim) / 2, class (x));


I thought about other solutions from the "real plain dump" integer world (like
C/C++) but these would not apply for Octave... This I am afraid to stay to
this solution.

What do you think about this Dan Sebald and Etienne de Foras?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54567>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/