[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Strange behaviour of median and std
From: |
Jaroslav Hajek |
Subject: |
Re: Strange behaviour of median and std |
Date: |
Wed, 17 Feb 2010 13:32:18 +0100 |
On Wed, Feb 17, 2010 at 12:53 PM, Marius Roets <address@hidden> wrote:
> Hi everybody,
>
> It took me an hour to get to the bottom of this. Can somebody please explain
> the output of this code to me.
>
> b = [255 255 0; 255 255 255];
> disp(b);
> median(b(:))
> std(b(:))
> b = uint8(b)
> median(b(:))
> std(b(:))
>
> Output:
>
> 255 255 0
> 255 255 255
> ans = 255
> ans = 104.10
> b =
>
> 255 255 0
> 255 255 255
>
> ans = 128
> ans = 43
>
> Thanks
> Marius
>
The uint8 case is performed using integer arithmetics, which is
saturated and truncated to integers, hence the incorrect results.
An easy remedy is to convert integers to doubles in "center". median
is already computed correctly in the development version.
http://hg.savannah.gnu.org/hgweb/octave/rev/e0767a0965f1
thanks
--
RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz