Re: [PATCH 1/2] tests: make test runner a script, not a shell function

[Stefano Lattarini]

On Monday 20 June 2011, Ralf Wildenhues wrote:
> * Jim Meyering wrote on Mon, Jun 20, 2011 at 10:29:50PM CEST:
> > Ralf Wildenhues wrote:
> > > One thing I've regularly done with new code that is not 100% backward
> > > compatible is have a new Automake option for them.  That is exactly why
> > > there is a 'parallel-tests': it is not fully compatible with the simple
> > > test driver, and requires work on behalf of developers using Automake.
> > 
> > Your call, but don't do it for me.
> > For things I maintain it'll be quick and easy to fix.
> 
> OK.
> 
> So, another thing to look out for, for Stefano: will the driver remain
> compatible to the current parallel-tests one,
>
It should, as much as possible.  It won't be truly 100% compatible, though;
but the incompatibilities should be quite small; to quote from the added
NEWS entry:

  - The parallel-tests driver is now implemented (partly at least) with
    the help of automake-provided auxiliary scripts (e.g., `test-driver'),
    instead of relying entirely on code in the generated Makefile.in.
    This has two noteworthy implications.  The first one is that projects
    using the `parallel-tests' option should now either run automake with
    the `--add-missing' option, or manually copy the `test-driver' script
    into their tree.  The second, and more important, implication is that
    now shell functions and non-environmental shell variables defined in
    TESTS_ENVIRONMENT are not anymore available in LOG_COMPILER and
    <ext>_LOG_COMPILER.  For example, it is now no more possible to define
    a shell function, say `custom_test_runner', in TESTS_ENVIRONMENT, and
    then setting "LOG_COMPILER = custom_test_runner" in the Makefile.am in
    order to use that function to run the tests.

Maybe we should also say that using TESTS_ENVIRONMENT to define a custom
test runner is now not only strongly deprecated (as it already was I hope),
but also unsupported and not working anymore?

Also, should I look for TESTS_ENVIRONMENT usages in google code search?
I was really hoping to spae myself the pain... ;-)

> and if not, is it possible to have a compatible one (without too much
> maintenance effort duplication)?  No need to go the effort right away.
>
Well, we could add a new option "old-parallel-tests" or something like
that, that causes the old code in 'check.am' (with few tweaks in order
to support the new parsing of `.log' files) to be used instead of the
'test-driver' script.  By refactoring some code in handle_tests(), we
could ensure not to add any real complexity to the automake script
(w.r.t. to my patches at least); but the duplication between 'check.am'
and 'test-driver' will unavoidable IMHO.

> I hope that we do not have to maintain four or more such drivers.
>
Me too.  And I also think we should start deprecating the old "serial"
driver ASAP (i.e., in 1.12, as I think 1.11.2 is sadly  tooearly); but
this is for another thread, and another time.

> Two
> is already more than nice for maintenance (and has caused sync issues
> already).  But maybe we can keep issues so small that we can deal with
> them in a NEWS entry after all.
> 
> Cheers,
> Ralf
> 

Regards,
  Stefano