[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: our benchmark-suite

From: Ludovic Courtès
Subject: Re: our benchmark-suite
Date: Wed, 02 May 2012 23:24:41 +0200
User-agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.93 (gnu/linux)


Neil Jerram <address@hidden> skribis:

> address@hidden (Ludovic Courtès) writes:
>>> My proposal is to rebase the iteration count in to run
>>> for 0.5s on some modern machine, and adjust all benchmarks to match,
>>> removing those benchmarks that do not measure anything useful.
>> Sounds good.  However, adjusting iteration counts of the benchmarks
>> themselves should be done rarely, as it breaks performance tracking like
>> <>.
>>> Finally we should perhaps enable automatic scaling of the iteration
>>> count.  What do folks think about that?
>>> On the positive side, all of our benchmarks are very clear that they are
>>> a time per number of iterations, and so this change should not affect
>>> users that measure time per iteration.
>> If the reported time is divided by the global iteration count, then
>> automatic scaling of the global iteration count would be good, yes.
> For I do still have all of the raw data
> including iteration counts, so I could easily implement dividing by the
> iteration count, and hence allow for future iteration count changes.
> Is there any downside from doing that?  (I don't think so.)

No, I guess.  And as you show, having raw data instead of synthesized
figures gives more freedom.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]