[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Tsp-devel] A test campaign manager for dtest
From: |
Frederik Deweerdt |
Subject: |
Re: [Tsp-devel] A test campaign manager for dtest |
Date: |
Mon, 14 Apr 2008 16:24:49 +0200 |
User-agent: |
Mutt/1.5.17 (2007-11-01) |
On Fri, Apr 11, 2008 at 11:01:18PM +0200, Eric Noulard wrote:
> 2008/4/11, Frederik Deweerdt <address@hidden>:
> > > The attached file details the main goals of the project. Please let us
> > know
> > > if you are interested in such an evolution inside dtest. Your potential
> > > remarks and/or propositions would of course be greatly appreciated.
> >
> > Nice proposal, thanks!
>
> +1
>
> >
> > I've got a few questions/remarks:
> >
> > - IMHO, this should be part of the dtest distribution
>
> I agree to but:
>
> I want to keep DTest as simple as possible.
>
> For example using DTest does not imply using a Test Campagn Manager
> DTest should be nicely layered such that lower blocks
> may be re-used.
>
> > - Would'nt YAML[1] be more flexible than XML? e.g. :
> >
> [...]
>
> > this would lessen the tests's burden of parameter parsing. We could in
> > particular avoid:
> > <parameter name="client_ip1" value="2.2.2.2"/>
> > <parameter name="client_ip2" value="2.2.2.2"/>
>
> I've no precise idea about that [yet]
>
> > - How does the campaign know about a particular test failing? Do we rely
> > on forwarding the test's TAP output?
>
> We discussed the fact that "DTest report" may be of different kind
> TAP was the first, Lionel already added MSC-like output.
I understand that most of us have no use of TAP output, but why not use
TAP-to-AnythingElse translators, that parse the DTestMaster's output,
so that inside dtest, we always use TAP?
> Independently of the "output format" each DTester should (generically)
> elaborate a test result. If an ok step has failed then DTester has failed
> then DTestmaster know it then Campaign Manager too.
>
> > In that case, the output needs to be
> > serialized it if several clients are running at the same time.
>
> Unless I miss your point I think DTestmaster already serialize
> the "distributed test" output?
>
> In fact there is no "real" serialize need since DTestmaster and
> all DTester runs on the same machine and are LOCALLY synchronized.
> However DTester have asynchronous "output" from the SSH Session Handler.
The case I was thinking of is as follows:
D Campaign Manager -> DTestMaster -> DTest sibling
Understands TAP <- TAP <- TAP
So it would make sense that the Campaign Manager sees:
TAP from client 1
TAP from client 1
TAP from client 1
TAP from client 2
TAP from client 2
TAP from client 2
Eventhough the lines where not received in that order by the
DTestMaster.
>
> The tests sequence itself may/should contains expect/barrier steps
> which ensure synchronization when (functionnalmy) needed.
Sure, my point is that the output needs to be serialized so that it's
easy to parse the TAP at the other end.
>
> I will be silent for a week, since I may not touch a keyboard during
> this time :=)
Have a nice time then!
Frederik
Re: [Tsp-devel] A test campaign manager for dtest, T. Decherf, 2008/04/14