Re: [PATCH] {GSoC} parallel-tests: simplify testsuite summary

[Stefano Lattarini]

On Thursday 07 July 2011, Ralf Wildenhues wrote:
>
> [SNIP longish discussion I'd like to keep for a later date]
>
> > You mean want them *even in the test-suite.log*?
> 
> Hmm.  --color is good enough, if it forces color on stdout, IMVHO.
>
Point is, it doesn't.  It only tells that the colors are to be be used
either if the user has forced them, or if he hasn't forbidden them and
using them seems sensible in the given scenario (i.e., TERM is not "dumb"
and stdout is a tty).  So `--maybe-colors' seems the best choice after
all.  Do you agree?

> > > Oh, is this user-influenceable at all btw?
> > >
> > For the testsuite summary that goes on stdout, yes, the user has complete
> > control over its colorizing (as he has for the colorizing of the testsuite
> > progress output); for the summary that goes in `test-suite.log', no, the
> > user can't colorize it, period.  This seems to me the most sensible policy.
> 
> OK.
> 

> > > Yeah, except now the summary looks _a lot_ like output from a single
> > > test.  Quite confusing for a simple parser (again, look at the autobuild
> > > scripts for an example).
> > >
> > But IMVHO, the leading `#' characters avoids confusion for the parser, and
> > the enclosing `=========...' lines avoid confusion for the humans.  Or not?
> 
> Maybe (see caveats above), but both require the "parser" to save state
> between lines.  Again, this is minor, but the easier to read, the better.
>
I'd rather not try to define the best format for the testsuite summary right
now.  I'd say instead that we temporary accept "my" new format, and that
we return on the issue later (surely before 1.12), with the following goals:

  - Keep the testsuite summary format easily parseable, both for our own
    tests (so that they will be easy to adjust) and for third party tools
    (which means that we have to decently document the format, and maybe
    offer scripts and tools to deal with it).

  - Keep the summary clean and pleasant, and avoid to make it overly
    obtrusive.

  - Ensure the format is easily extensible, and as much forward-compatible
    as possible.

Can we agree on this?

Thanks,
 Stefano