[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in

From: Rob Savoye
Subject: bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case
Date: Sat, 27 Jun 2020 06:49:30 -0600
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0

On 6/25/20 7:53 PM, Jacob Bachmeyer wrote:

> The original rationale for using UNRESOLVED here was that DejaGnu
> converts a test result to UNRESOLVED if too many errors or warnings were
> produced.  The DejaGnu manual indicates (section "A POSIX Compliant Test
> Framework") that UNRESOLVED is correct for a test where execution was
> interrupted or was set up incorrectly.  UNTESTED is specifically listed
> as a placeholder for an as-yet-unwritten testcase.  A "typical" GDB
> testsuite run has almost a hundred UNTESTED results but (without this
> patch) zero UNRESOLVED results.
> To me, this seems that UNRESOLVED is correct here, or that the manual
> has an error.

  So having written both, there is always the chance I've interpreted
things differently. If the test is interrupted, or has errors/warnings,
like timing problems for example, then it's UNRESOLVED. But a bug in Tcl
code enough to trigger unknown is UNTESTED, as the test never really
ran. I'd go with which definition the toolchain teams prefer, as this
effects validation testruns. I could go either way.

  I don't know if POSIX 1003.3 is still considered a standard, that was
a long time ago. The testsuites were primarily focused on XOpen and
POSIX conformance tests, not toolchains. I was
on the standards committee, so used them as a model cause they seemed a
good standard. But the details of the interpretation can now be whatever
we want it to be. We're probably the only ones left running TET
compliant testsuites anyway. :-)

  Anyway, whether a test run aborts on an error or not is not defined,
so it's up to us to decide.

        - rob -

reply via email to

[Prev in Thread] Current Thread [Next in Thread]