lmi
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [lmi] Automated GUI testing, revisited


From: Greg Chicares
Subject: Re: [lmi] Automated GUI testing, revisited
Date: Wed, 12 Nov 2014 12:18:19 +0000
User-agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0

On 2014-11-12 01:55Z, Greg Chicares wrote:
[...]
> Vadim--I believe I copied 'wx_test.conf' from an old email that
> you had sent. Was "time_disk=11000" an actual measurement? For
> single tasks, your machine should be faster than mine; so I'm
> wondering whether we're running the same test.
> 
> My "456ms" above looks a lot like "time_run" in 'wx_test.conf':
>   time_run=434
>   time_disk=11000
>   time_spreadsheet=710
> (I mention that without further investigation on my part).

Here's more information, from another run:

11:49:25: Run case for census_1: 416ms elapsed (expected -4.15%)
11:49:25:     benchmark_census: ERROR (Assertion 'std::fabs(diff_in_percents) < 
10' failed
(expected 11000ms, but actually took 416ms, i.e. -96.22%).
[file /lmi/src/lmi/wx_test_benchmark_census.cpp, line 101] )

Doesn't this suggest that the "ERROR" comparison is incorrect?
I can match "expected -4.15%" this way:
  (416-434)/434 = -0.04147...
so that seems right. Assuming that "416ms" is my "time_run",
I can match "i.e. -96.22%" this way:
  (416-11000)/11000 = -0.96218...
but it seems incorrect to compare observed "time_run" to
expected "time_disk", if that's what's really being done.

But I still think it's best to remove these relative-error
calculations, as suggested in the message I'm replying to.




reply via email to

[Prev in Thread] Current Thread [Next in Thread]