Re: [PATCH v7 18/21] simplebench/results_to_text: improve view of the table

[Max Reitz]

On 21.10.20 16:58, Vladimir Sementsov-Ogievskiy wrote:
> Move to generic format for floats and percentage for error.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>   scripts/simplebench/results_to_text.py | 13 ++++++++++++-
>   1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/simplebench/results_to_text.py 
> b/scripts/simplebench/results_to_text.py
> index 58d909ffd9..479f7ac1d4 100644
> --- a/scripts/simplebench/results_to_text.py
> +++ b/scripts/simplebench/results_to_text.py
> @@ -16,11 +16,22 @@
>   # along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   #
>   
> +import math
> +
> +
> +def format_value(x, stdev):
> +    stdev_pr = stdev / x * 100
> +    if stdev_pr < 1.5:
> +        # don't care too much
> +        return f'{x:.2g}'
> +    else:
> +        return f'{x:.2g} ± {math.ceil(stdev_pr)}%'

OK, so no magnitude-based precision this time (except for the %f -> %g 
change).  Works for me.

Other than that, I personally don’t like the relative standard deviation 
much, because the absolute SD immediately shows the 68 % boundaries (and 
an idea on the 95 % boundaries with 2σ), whereas the RSD just gives an 
impression on how spread out the data is.  (Which I find the absolute SD 
also does, when given together with the average, which is the case here.)

To be completely honest, though, I didn’t even know the term “relative 
SD” existed until a couple of minutes ago, and I didn’t know it was 
something that was used at all.
And if I haven’t seen the RSD used in practice, I can’t confidently say 
that I have good reasons not to like it.

But, well, I can’t have any confidence in liking it either, and because 
the change from ASD to RSD is basically the most important change of 
this patch (which I can’t really agree is an improvement), I can’t 
really give an R-b.

Perhaps this is OK:

Acked-by: Max Reitz <mreitz@redhat.com>

>   
>   def result_to_text(result):
>       """Return text representation of bench_one() returned dict."""
>       if 'average' in result:
> -        s = '{:.2f} +- {:.2f}'.format(result['average'], result['stdev'])
> +        s = format_value(result['average'], result['stdev'])
>           if 'n-failed' in result:
>               s += '\n({} failed)'.format(result['n-failed'])
>           return s