[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: An Emacs benchmarking suite

From: Phillip Lord
Subject: Re: An Emacs benchmarking suite
Date: Tue, 15 May 2018 14:24:58 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)

Stefan Monnier <address@hidden> writes:

>>> So, something like time to create a big list (for CPU), and time to read
>>> a defined file (for IO). Then you could say "this file should parse in
>>> 1x IO base-line + 10x CPU base-line.
>> You assume a linear scalability, but that is not necessarily so.
>> The ratio between performance indices of different codes could vary
>> depending on the build option and the underlying OS.
> Another issue is that performance measuring is notoriously difficult
> (even on an otherwise idle machine, let alone on some server that has
> other tasks running at the same time).  So you might be able to catch
> the "10x slower" case easily with a fairly high confidence that the
> problem is indeed that the code got slower, but if you want to catch the
> "20% slowdown" with any kind of confidence (without being drowned in
> false positives), you'll need either a very tight control on the test
> runs, or a good statistical analysis.

Indeed, it would be very broad brush. Or, alternatively, it would be
tied very tightly to a specific infrastructure.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]