Re: [PATCH v7 18/21] simplebench/results_to_text: improve view of the table

[Vladimir Sementsov-Ogievskiy]

13.11.2020 18:59, Max Reitz wrote:
> On 21.10.20 16:58, Vladimir Sementsov-Ogievskiy wrote:
> > Move to generic format for floats and percentage for error.
> >
> > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > ---
> >   scripts/simplebench/results_to_text.py | 13 ++++++++++++-
> >   1 file changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/simplebench/results_to_text.py 
> > b/scripts/simplebench/results_to_text.py
> > index 58d909ffd9..479f7ac1d4 100644
> > --- a/scripts/simplebench/results_to_text.py
> > +++ b/scripts/simplebench/results_to_text.py
> > @@ -16,11 +16,22 @@
> >   # along with this program.  If not, see <http://www.gnu.org/licenses/>.
> >   #
> > +import math
> > +
> > +
> > +def format_value(x, stdev):
> > +    stdev_pr = stdev / x * 100
> > +    if stdev_pr < 1.5:
> > +        # don't care too much
> > +        return f'{x:.2g}'
> > +    else:
> > +        return f'{x:.2g} ± {math.ceil(stdev_pr)}%'
>
> OK, so no magnitude-based precision this time (except for the %f -> %g change). 
>  Works for me.
>
> Other than that, I personally don’t like the relative standard deviation much, 
> because the absolute SD immediately shows the 68 % boundaries (and an idea on 
> the 95 % boundaries with 2σ), whereas the RSD just gives an impression on how 
> spread out the data is.  (Which I find the absolute SD also does, when given 
> together with the average, which is the case here.)

I realized that for myself, the only thing I need from +-deviation is a kind of 
"reliability" of the result. And it's more obvious for me in percentage. 
Looking at the table with a lot of numbers, where most of values have deviation less than 
+-5%, some values with deviation +-30 would catch the eye immediately. And with absolute 
SD I have to look through the table more carefully.

> To be completely honest, though, I didn’t even know the term “relative SD” 
> existed until a couple of minutes ago, and I didn’t know it was something that 
> was used at all.
> And if I haven’t seen the RSD used in practice, I can’t confidently say that I 
> have good reasons not to like it.
>
> But, well, I can’t have any confidence in liking it either, and because the 
> change from ASD to RSD is basically the most important change of this patch 
> (which I can’t really agree is an improvement), I can’t really give an R-b.
>
> Perhaps this is OK:
>
> Acked-by: Max Reitz <mreitz@redhat.com>

Thanks!

I don't have strict opinion about how this should look. For now these scripts 
don't have wide usage... We can always add an option to adjust table view, and 
I will not mind making absolute SD the default view.


--
Best regards,
Vladimir