[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [lmi] Benchmarking: gcc-8 beats gcc-10 soundly?

From: Greg Chicares
Subject: Re: [lmi] Benchmarking: gcc-8 beats gcc-10 soundly?
Date: Sat, 19 Sep 2020 16:55:41 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0

On 2020-09-19 15:48, Vadim Zeitlin wrote:
> On Sat, 19 Sep 2020 15:15:48 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
> GC> It looks like gcc-10 gives us slower lmi binaries. Picking
> GC> the third '--selftest' scenario as an index of performance
> GC> (results in microseconds--less is better):
> GC> 
> GC>      gcc-10   gcc-8  ratio
> GC>      ------   -----  -----
> GC>      102659   84947   1.21  32-bit
> GC>       50121   37410   1.34  64-bit
> GC> 
> GC> The fourth scenario is even worse:
> GC> 
> GC>       33250   20654   1.61  32-bit
> GC>       24616   13009   1.89  64-bit
> GC> 
> GC> Comparing to the benchmarks here:
> GC>   
> https://openbenchmarking.org/result/1912179-HU-COREI759632&obr_sgm=y&obr_vb=y&obr_sgm=y&obr_swl=y&obr_vb=y
> GC> lmi fares worse than the worst phoronix scenarios, i.e.,
> GC> "libgav1" and "function objects".
> GC> 
> GC> Vadim--Does this seem so astonishing that it can't be
> GC> true?
>  I've already seen performance regressions in newer g++ versions, but I
> don't think I've seen anything nearly like 89% slowdown, so it's indeed
> very astonishing. But I have trouble seeing how could it be not true, if
> you consistently obtain such results. And you're not the only one, see e.g.
> this bug report https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96337

Thanks, I hadn't searched bugzilla.

>  Unfortunately there is no clear conclusion there, as gcc developers can't
> reproduce the problem. They do say that -O2 has been changed in 10.x, so it
> could be worth using -O3 with it and see if it helps. Should I/we do it or
> will you test this yourself?

I already did it for 8.x, which needs
(because of -Werror); it didn't make much difference there.
I'll try it with 10.x soon.

>  BTW, while looking at this, I had another question: could we add the
> standard deviation or at least min/max time output to AliquotTimer? It
> might show that the code has indeed become uniformly slower, or that there
> are just more outliers. I don't know which scenario would actually be
> worse, but it looks like it would be useful to know which one are we
> dealing with. If you think this could be useful, I could make a patch
> adding this, of course -- just please let me know.

Sure, go right ahead. I have no objection to showing σ.
I'll probably want to twiddle the output format to make
it really compact, so that we can fit more information
on one line--very roughly, something like this:

-  1.323e-02 s mean;      13009 us least of  76 runs
+  μ=1.32e-02 σ=9.87e-01 min=13009us runs=76

which leaves room if you want to throw in kurtosis or
whatever. But let me worry about formatting, because
it's the math stuff that I think you really want to do
(Welford's algorithm or whatever) and a patch that
just adds some code to do those calculations is easy
to accept because it won't have any perceptible effect
until the format is changed to print it.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]