[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in
From: |
Rob Savoye |
Subject: |
bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case |
Date: |
Sat, 27 Jun 2020 06:49:30 -0600 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 |
On 6/25/20 7:53 PM, Jacob Bachmeyer wrote:
> The original rationale for using UNRESOLVED here was that DejaGnu
> converts a test result to UNRESOLVED if too many errors or warnings were
> produced. The DejaGnu manual indicates (section "A POSIX Compliant Test
> Framework") that UNRESOLVED is correct for a test where execution was
> interrupted or was set up incorrectly. UNTESTED is specifically listed
> as a placeholder for an as-yet-unwritten testcase. A "typical" GDB
> testsuite run has almost a hundred UNTESTED results but (without this
> patch) zero UNRESOLVED results.
>
> To me, this seems that UNRESOLVED is correct here, or that the manual
> has an error.
So having written both, there is always the chance I've interpreted
things differently. If the test is interrupted, or has errors/warnings,
like timing problems for example, then it's UNRESOLVED. But a bug in Tcl
code enough to trigger unknown is UNTESTED, as the test never really
ran. I'd go with which definition the toolchain teams prefer, as this
effects validation testruns. I could go either way.
I don't know if POSIX 1003.3 is still considered a standard, that was
a long time ago. The testsuites were primarily focused on XOpen and
POSIX conformance tests, not toolchains. I was
on the standards committee, so used them as a model cause they seemed a
good standard. But the details of the interpretation can now be whatever
we want it to be. We're probably the only ones left running TET
compliant testsuites anyway. :-)
Anyway, whether a test run aborts on an error or not is not defined,
so it's up to us to decide.
- rob -
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, (continued)
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Jacob Bachmeyer, 2020/06/26
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Pedro Alves, 2020/06/27
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Pedro Alves, 2020/06/27
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Jacob Bachmeyer, 2020/06/27
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Pedro Alves, 2020/06/27
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Jacob Bachmeyer, 2020/06/30
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Pedro Alves, 2020/06/30
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Jacob Bachmeyer, 2020/06/30
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Rob Savoye, 2020/06/25
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Jacob Bachmeyer, 2020/06/25
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case,
Rob Savoye <=
- bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case, Jacob Bachmeyer, 2020/06/27
bug#41824: Branch PR41918 containing fixes for these issues is into final testing, Jacob Bachmeyer, 2020/06/30