[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: How do you set VERBOSE for parallel testin

From: Arthur Schwarz
Subject: RE: How do you set VERBOSE for parallel testin
Date: Tue, 12 May 2015 15:42:15 -0700

> >>
> >> If any test causes a FAIL, XPASS, or ERROR condition, then 'make check'
> >> will likewise fail.  The output from make is telling you that your
> >> testsuite caught failures that should not be present if the testsuite
> >> were passing.
> >    
> >   My issue is that the output indicates that the rule for test-suite.log
> > fails. Is this expected? My thought (from the automake manual) is that
> > if there is a failure there should be no difficulty creating or
> > test-suite.log. The test harness created a test-suite.log (included as
> > attachment) but failed to output it. Don't know why.
> Yes, this is expected.  test-suite.log is created no matter what, but
> the rules associated with creating it fail if the log contains any
> failure reports, so that make will quit running and let you investigate
> those failures.

If I understand this correctly, when executing test scripts the first
failure will cause make to discontinue execution (?). This doesn't happen.
Make continues until all tests in the variable TESTS are executed. Then
again, I must be misunderstanding you?

> As for why test-suite.log was not replayed as part of the make output,
> I'm not sure that you had VERBOSE=1 set.  So far, you've only attached
> the generated Makefile (but not the source, and didn't show
> what command you ran.  You'll need to make your setup easier for others
> to reproduce before we can see what you are attempting.

I am including several files.
   test.pdf        console output showing test results     src/Makefile
   Test-suite.log  generated file    ./     ./
> As for the test-suite.log not having much output, it is probably because
> your tests are not very noisy.  Running make captured all of the output
> of your tests, but if your tests didn't output anything, then the log
> has very little to show.  Again, without knowing what your test scripts
> are doing, it's hard to reproduce your setup.

>From the Automake Manual Section 15.2.3 Parallel Test Harness, " The output
from failed tests is collected in the test-suite.log file. If the variable
'VERBOSE' is set, this file is output after the summary."

This quote is in fact not fully accurate. The output from all tests are
collected in the test-suite.log file. I can not set VERBOSE to output the
test-suite.log file.

Your statement concerning the paucity of output does not preclude showing
any output. In fact, removing the generated "echo" statements from the
generated shell scripts in, and making no other change, will
show that the test harness generates an output for each test case executed
in the test-suite.log file. 

I have tried to set VERBOSE in my to '1' and to 'yes' and "yes"
and just plain yes. The test-suite.log file does not output. Am I missing
something? How is VERBOSE supposed to be turned on?

Anyhow, I'm puzzled as to what to do and how to check this out. The general
comments probably apply to all variables which have a 'yes/no' value. What
should the value be to turn them on or off? A similar question occurs in
setting AM_COLOR_TEST to 'always'. What are the legitimate values?

Sorry about the delay in answering.

Attachment: test.pdf
Description: Adobe PDF document

Description: Binary data

Attachment: test-suite.log
Description: Binary data

Description: Binary data

Description: Binary data

reply via email to

[Prev in Thread] Current Thread [Next in Thread]