[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v4 1/3] parallel-tests: add auxiliary script 'pt-driver', ref

From: Stefano Lattarini
Subject: Re: [PATCH v4 1/3] parallel-tests: add auxiliary script 'pt-driver', refactor
Date: Sun, 19 Jun 2011 12:40:37 +0200
User-agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; )

On Friday 17 June 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Thu, Jun 16, 2011 at 10:00:31AM CEST:
> > This refactoring should cause no API of functionality change,
> > and is meant only to simplify the future implementation of TAP
> > and SubUnit testsuite drivers.  More precisely, our roadmap is
> > to move most of the "testsuite driving" features out of the
> > Automake-generated Makefiles, and into external scripts with
> > well-defined interfaces.  This will allow the user to define
> > its own personalized testsuite drivers, and will also offer us
> > a framework upon which to implement our new TAP and SubUnit
> > drivers, all in a very unobtrusive way and retaining an high
> > degree of code reuse and backward-compatibility.
> I generally like the direction this is taking.  The point of best
> separation between which code goes into and which into
> the driver scripts can be fine-tuned when we have more than one such
> script.
> Actually, yes, before deciding on this for real I really do want to see
> a nontrivial other driver script.  There is no point in hardcoding
> too much in several driver scripts if it all needs to be the same
> anyway.
> Please measure the time overhead your changes introduce into the current
> code, for a trivial testsuite (say, 50 tests running 'true'), and a
> nontrivial one like Automake's and one with faster tests.  Thanks.

I've run each of the testsuites repeated times, in the hope of shaking
out most of the noise.

I've run the Automake testsuite four times on a fast Solaris 10 system
(quadcore 3 GHz, 20 Gb of RAM) in low priority (a niceness of 19) and
with 32 processes concurrently ("make -j 32 check")

Then, I've run the Coreutils testsuite three times, on my Debian desktop
(single core 1.5 Ghz, 768 Mb of RAM), with one job at the time (i.e.,
plain "make check", no `-j' option).

Finally, I've run the "dummy" testsuite on my desktop ten times -- five
in serial mode ("make check") and five in concurrent mode, with four jobs
at the time ("make -j4 check").


Here are the results:

Automake Testsuite

  master branch

  real    76m46.661s
  user    127m54.170s
  sys     136m31.526s

  test-harness branch

  real    77m42.103s
  user    126m7.108s
  sys     140m26.200s

Coreutils Testsuite

  master branch

  real    34m55.552s
  user    4m23.216s
  sys     6m20.312s

  test-harness branch

  real    35m42.560s
  user    4m29.453s
  sys     6m27.740s

Dummy Testsuite

  master branch

  real    1m3.158s
  user    0m11.045s
  sys     0m17.421s

  test-harness branch

  real    1m11.025s
  user    0m16.249s
  sys     0m21.073s



reply via email to

[Prev in Thread] Current Thread [Next in Thread]