Re: An Emacs benchmarking suite

[Eli Zaretskii]

> From: address@hidden (Phillip Lord)
> Cc: "Siraphob \(Ben\) Phipathananunth" <address@hidden>,  address@hidden,  
> address@hidden
> Date: Mon, 14 May 2018 12:37:09 +0100
> 
> > Performance should indeed compare several versions n the same system.
> 
> 
> I wonder if there is a way to work out a base line value. Benchmarks
> could then look for multiples of this.

But such a base line would also have to be specific to a platform and
a given set of build options.

> So, something like time to create a big list (for CPU), and time to read
> a defined file (for IO). Then you could say "this file should parse in
> 1x IO base-line + 10x CPU base-line.

You assume a linear scalability, but that is not necessarily so.  The
ratio between performance indices of different codes could vary
depending on the build option and the underlying OS.

Btw, IME I/O is mostly negligible in Emacs applications, and generally
is not interesting for the issue at hand.  Only CPU is important, and
maybe also memory usage.