[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-bug-tracker] [bug #54567] median(uint8) gives bad results

From: Kai Torben Ohlhus
Subject: [Octave-bug-tracker] [bug #54567] median(uint8) gives bad results
Date: Fri, 24 Aug 2018 18:54:15 -0400 (EDT)
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

Follow-up Comment #2, bug #54567 (project octave):

Matlab R2018a returns "140" in both cases of the original comment.

The patch file #44857 of comment #1 works

-    retval = sum (nth_element (x, k:k+1, dim), dim, "native") / 2;
+    retval = mean (nth_element (x, k:k+1, dim), dim, "native");

But I think it makes things more complicated and maybe slower for large data?!
 Basically, mean() would perform the computation with double precision and
finally convert the result to the input data type.  Why not doing the very
same without additional function calls in one line and tidy up the nasty
"logical" exception:

-    retval = sum (nth_element (x, k:k+1, dim), dim, "native") / 2;
-    if (islogical (x))
-      retval = logical (retval);
-    endif
+    ## Perform the averaging in double precision to avoid intermediate
+    ## overflows of finite data types like "int8".
+    retval = cast (sum (nth_element (x, k:k+1, dim), dim) / 2, class (x));

I thought about other solutions from the "real plain dump" integer world (like
C/C++) but these would not apply for Octave... This I am afraid to stay to
this solution.

What do you think about this Dan Sebald and Etienne de Foras?


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]