[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 00/17] De-recursion for the 'tests' subdirectory.

From: Stefano Lattarini
Subject: Re: [PATCH 00/17] De-recursion for the 'tests' subdirectory.
Date: Tue, 04 Sep 2012 12:39:29 +0200

Hi Jim.

On 09/04/2012 12:13 PM, Jim Meyering wrote:
> Jim Meyering wrote:
>> Stefano Lattarini wrote:
>>> Plus some other preparatory, follow-up or "while-at-it" changes.
>>> After this, "make syntax-check" and "make distcheck" still pass on
>>> my Debian box.
>>> This completes the first important part of the "de-recursion" of the
>>> coreutils build system: the compilation of the programs, the creation
>>> of Texinfo documentation and manpages, and the testsuite are now all
>>> controlled by one top-level Makefile.
>>> The next step would be de-recursing the gnulib-imported 'lib/' and
>>> 'gnulib-tests/' subdirectories, but I'll leave that to someone else ;-)
>> Thanks a lot.
>> I've applied these and ensured that distcheck works for me, too.
>> (BTW, in coreutils, distcheck includes "make syntax-check" -- at least
>> it used to)
>> I hope to actually review the changes later today.
>> The only missing pieces:
>>   - a timing comparison: does this result in a performance improvement?
>>   - a NEWS entry: every 70+-change-set deserves one ;-)
>>     One incidental improvement I've noticed: it fixes at least one bug
>>     that would cause an occasional parallel "make syntax-check" failure.
> Hi Stefano,
> I still haven't reviewed your changes, but in using them I did notice
> a missing dependency:
> I ran this command to run the single test:
>     $ make tests/rm/v-slash.log
>     PASS: tests/rm/v-slash.log
> then modified remove.c and recompiled src/rm and went to run the test again:
>     $ make tests/rm/v-slash.log
>     make: `tests/rm/v-slash.log' is up to date.
> Oops.  It didn't run.  The individual test scripts (the .log witness files,
> actually) do not depend on the programs they test.
The above usage is not actually supported Automake interface (for one thing,
it won't honour $(check_PROGRAMS), nor $(BUILT_SOURCES)).  To run a subset
of tests, you should do this:

    $ make check TESTS="tests/rm/v-slash"

And to run them lazily (i.e., only if one of their prerequisites has
changed), you should do:

    $ make check TESTS="tests/rm/v-slash" RECHECK_LOGS=

And if you want to re-run a test without overwriting the pre-existing
test-suite.log, here is what you should do:

    $ make check TESTS="tests/rm/v-slash" TEST_SUITE_LOGS=partial.log

All of this should be explained in the Automake manual, subsection
15.2.3, "Parallel Test Harness":


> From 45bf1ab13682f8842b2b9ed81bb9069f6e5b01b3 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <address@hidden>
> Date: Tue, 4 Sep 2012 12:12:19 +0200
> Subject: [PATCH] tests: depend on the programs we're testing
> * tests/ ($(TEST_LOGS)): Depend on $(PROGRAMS),
Beware that $(PROGRAMS) is not a documented Automake interface either.

> so that tests are rerun when any program is rebuilt.
Apart from my nits above, I believe this is a good idea, because it will
cause the documented interface for "lazy re-runs":

    $ make check TESTS="tests/rm/v-slash" RECHECK_LOGS=

to DTRT.

Your idiom won't be completely precise though, since $(TEST_LOGS) can be
overridden at make runtime by the user (either directly, or as a consequence
of his overriding of $(TESTS)); but being 100% correct would require you to
duplicate some of messy logic Automake applies to $(TESTS), to apply it to
$(all_tests) instead:

    all_test_logs1 = $(all_tests:=.log)
    all_test_logs2 = $(
    all_test_logs3 = $(
    all_test_logs = $(all_test_logs3:.xpl.log=.log)
    $(all_test_logs): $(PROGRAMS)

but that's not worth the hassle IMHO.

> Technically, we could
> specify precisely which few programs are dependents of each test,
> but that can come later, if deemed worth the trouble and maintenance
> burden.  Also, there is the issue of the primary program(s) being
> tested (i.e., those itemized via print_ver_) versus those that are
> tested incidentally: for example, nearly every test exercises "rm"
> when its clean-up code removes files.
> ---
>  tests/ | 2 ++
>  1 file changed, 2 insertions(+)
> diff --git a/tests/ b/tests/
> index b129475..2440667 100644
> --- a/tests/
> +++ b/tests/
> @@ -784,3 +784,5 @@ pr_data =                                 \
>    tests/pr/ttb3-0FF                          \
>    tests/pr/ttb3-FF                           \
>    tests/pr/w72l24f-ll
> +
> --


reply via email to

[Prev in Thread] Current Thread [Next in Thread]