[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-dejagnu] dejagnu-report-card (was: BUG: improper format string

From: Jacob Bachmeyer
Subject: Re: [Bug-dejagnu] dejagnu-report-card (was: BUG: improper format string construction in framework.exp)
Date: Mon, 05 Nov 2018 23:26:18 -0600
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20090807 MultiZilla/ SeaMonkey/1.1.17 Mnenhy/

Ben Elliston wrote:
On Tue, Oct 30, 2018 at 08:58:46PM -0500, Jacob Bachmeyer wrote:
Lastly, is a soft dependency on GNU awk acceptable for DejaGnu or
would I need to rewrite my testsuite summarization tool in Tcl or
Expect before contributing it for use with the DejaGnu testsuite?

I am happy for contributed tools like this to be written in GNU
awk. Please try and use portal AWK if you can, but if not, then
perhaps give the file a .gawk extension to make it clear that you need
GNU awk?

The AWK predecessor (written as part of a testsuite for another project that I intend to offer to GNU after the lowest-level basic functionality is ready; other parts of that testsuite already relied on GNU awk when this tool was written) uses both asort and asorti, so some work will be needed to get it into portable AWK. Reading sum2junit.sh gave me a few ideas, and I have written both shell and Tcl versions of a tool that I am now calling "dejagnu-report-card".

An attempt to use Expect to read DejaGnu summary files had extremely poor performance. The shell and Tcl versions have similar performance; one spends more time in the system and the other spends more time in user mode, but overall, they seem to have similar performance. The AWK predecessor blows them both away, usually completing in about 1/2 (on the DejaGnu testsuite) to 1/5 (on a larger testsuite with ~2000 results) of the time. (This is not noticeable on the DejaGnu testsuite, but ~2000 unit test results can take about 200ms for the shell and Tcl versions to read on my system.)

Also, I would like to see this be more than another tool in contrib; I plan to adapt the "runtest" shell script into a "dejagnu" multi-purpose launcher, supporting auxiliary commands in a style similar to git(1). (The runtest(1) command itself would remain "for historical reasons".) As such, "dejagnu report-card" would be the first such command added, with future plans including some kind of "canned test" feature to help with creating new testsuites and adding tools. Something like "dejagnu create testsuite" and "dejagnu add tool" would be nice. These will need careful review for portability, since I only have GNU close at hand for testing.

With this approach, including multiple versions and selecting at configure time which version to install should not be a problem. (Have GNU awk? Install the .gawk version. Have only some other AWK? Install the portable .awk version. Have no AWK? Install the Tcl version. The dejagnu(1) launcher would adapt at run-time.)

This leads to a slightly more complex "structure" for DejaGnu, with an "inner core" (runtest(1) and its components) and an "outer core" (the dejagnu(1) subcommands) that is also installed. One of the reasons for this is that I would like the DejaGnu testsuite itself to use the dejagnu-report-card tool, and possibly for a future version of Automake to use it if the DEJATOOL variable indicates to test multiple tools.

Here's what the output looks like:

  runtest  |      77      0      0      0      0      0      0
libdejagnu  |       5      0      0      0      0      0      0
  testrun  |       0      0      0      0      0      0      0
           |      82      0      0      0      0      0      0

The "testrun" line will be eliminated as part of planned improvements to the DejaGnu testsuite.

-- Jacob

[PS: A batch of your emails seems to have been "stuck" somewhere along the way for the past five days and have just now arrived in my inbox and at the list archives.]

reply via email to

[Prev in Thread] Current Thread [Next in Thread]