[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in

From: Jacob Bachmeyer
Subject: bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case
Date: Fri, 26 Jun 2020 17:58:09 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20090807 MultiZilla/ SeaMonkey/1.1.17 Mnenhy/

Pedro Alves wrote:
On 6/26/20 10:53 AM, Pedro Alves wrote:
to repeat at the very end of a test run with a big warning about test cases 
that crashed?
I did not suggest to repeat anything.

I misunderstood you here -- I somehow thought that that by "repeat" you meant
re-running the testcase.

No, repeating output of at least the name of the file, and possibly the entire error dump.

Still, I would not suggest to repeat the error.  Let me clarify -- with this:


Now, you already have an indication that something went wrong
because make existed with error.  But people may miss that.

They will easily miss that -- DejaGnu exits with a failure status if any tests produced "surprising" results: FAIL, XPASS, or UNRESOLVED. The GDB testsuite typically has multiple FAIL results, so exiting with an error code is routine.

The issue for me is that the "gdb Summary" tally did not
mention the aborted testcase.  I would like to see something
like this instead:

 # of expected passes            208
 # of nasty OMG FIX! tcl errors    1


Your UNRESOLVED is of course better than the status quo.

UNRESOLVED is a step in the right direction, but I agree that more is needed. We are somewhat limited by POSIX, which does not define a separate result type for "test case crashed" and seems to roll that into UNRESOLVED, although POSIX appears to require a testsuite to have a strictly-defined set of tests and that a run must produce results for exactly all of them, which requires information DejaGnu does not normally have about the running testsuite. So, while UNRESOLVED is needed, we could also maintain a separate count of UNRESOLVED-due-to-crash, or simply store away the error message/errorCode/errorInfo tuples and emit them a second time at the end of the run, also easy in Tcl 8.

I am leaning towards this latter solution precisely because it would be loud and obnoxious, while still maintaining a professional tone. (Listing a count of "nasty OMG FIX! >:-( tcl errors", while certainly expressing the proper intent, probably should not find its way into a DejaGnu release, nor should an ASCII-art C'thulu reminding the user that broken testsuites drive programmers to madness, complete with randomly-chosen "your remaining sanity" score... :-) ) Repeating error dumps at the end of the run also fits well into an XML log model, where the collected errors could go into their own structure at the end of the file, without complicating the tag model used for the main test results. Simple importers would still get the inserted UNRESOLVED result, while more thorough or specialized tools could analyze the errors.

-- Jacob

reply via email to

[Prev in Thread] Current Thread [Next in Thread]