[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: our benchmark-suite

From: Neil Jerram
Subject: Re: our benchmark-suite
Date: Sat, 28 Apr 2012 23:09:35 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux)

address@hidden (Ludovic Courtès) writes:

>> My proposal is to rebase the iteration count in to run
>> for 0.5s on some modern machine, and adjust all benchmarks to match,
>> removing those benchmarks that do not measure anything useful.
> Sounds good.  However, adjusting iteration counts of the benchmarks
> themselves should be done rarely, as it breaks performance tracking like
> <>.
>> Finally we should perhaps enable automatic scaling of the iteration
>> count.  What do folks think about that?
>> On the positive side, all of our benchmarks are very clear that they are
>> a time per number of iterations, and so this change should not affect
>> users that measure time per iteration.
> If the reported time is divided by the global iteration count, then
> automatic scaling of the global iteration count would be good, yes.

For I do still have all of the raw data
including iteration counts, so I could easily implement dividing by the
iteration count, and hence allow for future iteration count changes.

Is there any downside from doing that?  (I don't think so.)


reply via email to

[Prev in Thread] Current Thread [Next in Thread]