Re: Time to merge scratch/correct-warning-pos into master, perhaps?

[Eli Zaretskii]

> Date: Wed, 26 Jan 2022 18:41:57 +0000
> From: Gregory Heytings <gregory@heytings.org>
> Cc: mattiase@acm.org, larsi@gnus.org, emacs-devel@gnu.org,
>  monnier@iro.umontreal.ca, acm@muc.de
> 
> > I think we need more measurements in scenarios closer to actual Emacs 
> > usage.  One is intensive display operation -- I think Alan posted 
> > something to that effect.  Another could be starting Gnus to read a 
> > large newsgroup.  Yet another could be reindent a large piece of C or 
> > Python or Lisp code.  Perhaps also "M-x occur" through a large buffer. 
> > Stuff like that -- any command that tends to be used frequently and is 
> > known to take a tangible amount of time.
> >
> 
> The problem is that, IME, such benchmarks do not show anything when the 
> slowdown is not significant enough (> 10%), because the measurements vary 
> a lot depending on external factors.

Which external factors are those?

> I wonder how Alan could conclude that there is a slowdown of "less
> than 1%", when I run his benchmark on my (otherwise unloaded)
> computer, the running times I get are anywhere between 17s and 20s.
> Taking the average of such measures does not really make sense.

That is not my experience: my measurements of these and other
benchmarks are remarkably stable, as long as no heavy job (such as
building some large package) is running in parallel.

> However, I found one measurement that seems to give reasonably stable 
> results.  On my Debian bookworm computer, with a standard build, on 
> src/xdisp.c, (benchmark-run 100 (occur "a.b")) needs on average (with 10 
> runs) 7.42 seconds (standard deviation 0.04) on 3b33a14380 and 7.66 
> seconds (standard deviation 0.05) on 7922131bb2.  That's a >3% slowdown.

OK, thanks.