[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: An Emacs benchmarking suite
From: |
Phillip Lord |
Subject: |
Re: An Emacs benchmarking suite |
Date: |
Tue, 15 May 2018 14:24:58 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) |
Stefan Monnier <address@hidden> writes:
>>> So, something like time to create a big list (for CPU), and time to read
>>> a defined file (for IO). Then you could say "this file should parse in
>>> 1x IO base-line + 10x CPU base-line.
>>
>> You assume a linear scalability, but that is not necessarily so.
>> The ratio between performance indices of different codes could vary
>> depending on the build option and the underlying OS.
>
> Another issue is that performance measuring is notoriously difficult
> (even on an otherwise idle machine, let alone on some server that has
> other tasks running at the same time). So you might be able to catch
> the "10x slower" case easily with a fairly high confidence that the
> problem is indeed that the code got slower, but if you want to catch the
> "20% slowdown" with any kind of confidence (without being drowned in
> false positives), you'll need either a very tight control on the test
> runs, or a good statistical analysis.
Indeed, it would be very broad brush. Or, alternatively, it would be
tied very tightly to a specific infrastructure.
Phil
- Re: An idea, now that we have dynamic loading, (continued)
- Re: An idea, now that we have dynamic loading, John Wiegley, 2018/05/10
- Re: An idea, now that we have dynamic loading, Eli Zaretskii, 2018/05/11
- Re: An idea, now that we have dynamic loading, John Wiegley, 2018/05/11
- Re: An idea, now that we have dynamic loading, Siraphob (Ben) Phipathananunth, 2018/05/11
- Re: An idea, now that we have dynamic loading, Eli Zaretskii, 2018/05/11
- An Emacs benchmarking suite (was: An idea, now that we have dynamic loading), Siraphob (Ben) Phipathananunth, 2018/05/11
- Re: An Emacs benchmarking suite (was: An idea, now that we have dynamic loading), Eli Zaretskii, 2018/05/11
- Re: An Emacs benchmarking suite, Phillip Lord, 2018/05/14
- Re: An Emacs benchmarking suite, Eli Zaretskii, 2018/05/14
- Re: An Emacs benchmarking suite, Stefan Monnier, 2018/05/14
- Re: An Emacs benchmarking suite,
Phillip Lord <=
Re: An idea, now that we have dynamic loading, Tom Tromey, 2018/05/10
- Re: An idea, now that we have dynamic loading, John Wiegley, 2018/05/10
- Re: An idea, now that we have dynamic loading, Tom Tromey, 2018/05/14
- Re: An idea, now that we have dynamic loading, John Wiegley, 2018/05/14
- Re: An idea, now that we have dynamic loading, Tom Tromey, 2018/05/21
- Re: An idea, now that we have dynamic loading, John Wiegley, 2018/05/21
- Re: An idea, now that we have dynamic loading, Tom Tromey, 2018/05/22