[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Autotest function usage
From: |
Ralf Wildenhues |
Subject: |
Re: Autotest function usage |
Date: |
Thu, 15 Nov 2007 22:24:18 +0100 |
User-agent: |
Mutt/1.5.13 (2006-08-11) |
* Eric Blake wrote on Thu, Nov 15, 2007 at 04:25:00AM CET:
> Timings on Cygwin; best of three (except for the time-consuming -100 test).
>
> Pre patch:
> $ \time ./testsuite 1
> 14.26user 6.09system 0:14.08elapsed 144%CPU (0avgtext+0avgdata
> $ \time ./testsuite -10
> 69.20user 27.79system 1:06.96elapsed 144%CPU (0avgtext+0avgdata
> $ \time ./testsuite -100
> 1223.11user 495.20system 18:54.70elapsed 151%CPU (0avgtext+0avgdata
> Post patch:
> $ \time ./testsuite 1
> 13.66user 5.35system 0:13.42elapsed 141%CPU (0avgtext+0avgdata
> $ \time ./testsuite -10
> 66.45user 28.01system 1:10.71elapsed 133%CPU (0avgtext+0avgdata
> $ \time ./testsuite -100
> 1162.81user 488.71system 18:14.99elapsed 150%CPU (0avgtext+0avgdata
> Cygwin gained 4% on both a single test and 10 tests, and gained 5.9% on
> the 100 tests (although that number is a bit skewed, since many of the
> first 100 tests include a microsuite that runs a single test, so you get
> some of the effects of the 1-test speedup or penalty for each of tests
> 53-100).
Looks to me like the patch regresses on 10 tests and improves on 1 and
100, rather than an improvement everywhere. Since the improvement on
100 tests is quite noticable, I guess that's ok.
> > Do you think this is worthwhile to apply?
>
> Yes, after the nits in my previous mail are addressed. (A shell script
> that emits an awk script that emits a shell script that creates files for
> sourcing, all because it is faster than making the original shell do all
> the parsing. What an interesting concept.)
Yep.
> > +# Create a test file that has more than 99 words in a line, for Solaris
> > awk.
> > +# While at that, try out the limit of 2000 bytes in a text file line.
> > +
> > +AT_CHECK_AT_TEST([Long test source lines],
> > +[m4_for([nnn], [1], [999], [], [: ])
> > +AT_CHECK([:])
> > +])
>
> Cute.
Well, I checked again, this test isn't quite yet what I wanted. I would
like to prevent us from accidentally inducing awk to split the input.
Right now, there would be no need for a test to this end: Autoconf's
testsuite itself will fail to be parsed correctly by awk due to the
| at_groups_all=' 1 2 3 ... '
line which has many more than 99 words. However, this part of the code
is due for a rewrite soon anyway, as it's bound to exceed portable text
file line length eventually.
The `Long test source lines' will keep us from regressing even then.
But for that, let's also get the awk script to run over the test in the
non-emit state: run it with the test disabled (-k skipalltests).
So I applied like this.
Cheers,
Ralf
2007-11-15 Paolo Bonzini <address@hidden>
and Ralf Wildenhues <address@hidden>
* lib/autotest/general.m4 (at_func_test): Remove.
(AT_INIT): Pre-extract test groups into separate files.
(AT_CLEANUP): Source pre-extracted file instead of calling at_func_test.
Remove at-test-source files together with the $at_group_dir.
* tests/autotest.at (Long test source lines): New test.
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 4d92e6d..fd39ae9 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -185,6 +185,7 @@ m4_define([_AT_NORMALIZE_TEST_GROUP_NUMBER],
# Begin test suite.
m4_define([AT_INIT],
[m4_pattern_forbid([^_?AT_])
+m4_pattern_allow([^_AT_T_EOF$])
m4_define([AT_TESTSUITE_NAME],
m4_defn([AT_PACKAGE_STRING])[ test suite]m4_ifval([$1], [: $1]))
m4_define([AT_ordinal], 0)
@@ -310,15 +311,6 @@ at_func_diff_devnull ()
$at_diff "$at_devnull" "$[1]"
}
-# at_func_test NUMBER
-# -------------------
-# Parse out test NUMBER from the tail of this file.
-at_func_test ()
-{
- sed -n
'/address@hidden:@AT_START_'$[1]'$/,/address@hidden:@AT_STOP_'$[1]'$/p'
"$at_myself" \
- > "$at_test_source"
-}
-
# at_func_create_debugging_script
# -------------------------------
# Create the debugging script $at_group_dir/run which will reproduce the
@@ -431,7 +423,7 @@ at_status_file=$at_suite_dir/at-status
at_stdout=$at_suite_dir/at-stdout
at_stder1=$at_suite_dir/at-stder1
at_stderr=$at_suite_dir/at-stderr
-# The file containing the function to run a test group.
+# The stem for files containing a test group.
at_test_source=$at_suite_dir/at-test-source
# The file containing dates.
at_times_file=$at_suite_dir/at-times
@@ -871,6 +863,36 @@ else
at_diff=diff
fi
+{
+ echo 'BEGIN {'
+ for at_group in $at_groups; do
+ at_group_normalized=$at_group
+ _AT_NORMALIZE_TEST_GROUP_NUMBER(at_group_normalized)
+ echo " outfile[[\"$at_group\"]] =
\"$at_test_source-$at_group_normalized\""
+ done
+ AS_ECHO(['}
+emit == 0 && /address@hidden:@AT_START_/ {
+ test = substr($ 0, 11);
+ if (outfile[[test]]) {
+ emit = 1
+ print "cat >\"" outfile[[test]] "\" <<'\''_AT_T_EOF'\''"
+ }
+}
+emit != 0 && /address@hidden:@AT_STOP_/ {
+ print "_AT_T_EOF"
+ emit = 0
+}
+emit != 0 { print }
+'])
+} > "$at_test_source.awk"
+
+# Extract test group that will be run from the tail of this file
+if awk -f "$at_test_source.awk" "$at_myself" >"$at_test_source.sh" \
+ && . "$at_test_source.sh" \
+ && rm -f "$at_test_source.awk" "$at_test_source.sh"; then :; else
+ AS_ECHO(["$as_me: unable to parse test groups"]) >&2
+ exit 1
+fi
m4_text_box([Driver loop.])
for at_group in $at_groups
@@ -906,10 +928,7 @@ do
at_tee_pipe='cat >> "$at_group_log"'
fi
- if at_func_test $at_group && . "$at_test_source"; then :; else
- AS_ECHO(["$as_me: unable to parse test group: $at_group"]) >&2
- at_failed=:
- fi
+ . "$at_test_source-$at_group_normalized"
# Be sure to come back to the suite directory, in particular
# since below we might `rm' the group directory we are in currently.
@@ -973,9 +992,12 @@ _ATEOF
# Cleanup the group directory, unless the user wants the files.
if $at_debug_p ; then
at_func_create_debugging_script
- elif test -d "$at_group_dir"; then
- find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
- rm -fr "$at_group_dir"
+ else
+ if test -d "$at_group_dir"; then
+ find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
+ rm -fr "$at_group_dir"
+ fi
+ rm -f "$at_test_source-$at_group_normalized"
fi
;;
*)
diff --git a/tests/autotest.at b/tests/autotest.at
index 54e7464..fa7b7ea 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -335,6 +335,21 @@ AT_CHECK_AT_TITLE_CHAR([Longer test title],
[01234567890123456789012345678901234], [], [], [54])
+## ----------------------- ##
+## Long test source lines. ##
+## ----------------------- ##
+
+# Create a test file that has more than 99 words in a line, for Solaris awk.
+# While at that, try out the limit of 2000 bytes in a text file line.
+
+AT_CHECK_AT_TEST([Long test source lines],
+[m4_for([nnn], [1], [999], [], [: ])
+AT_CHECK([:])
+], [], [], [], [ignore],
+[AT_CHECK([$CONFIG_SHELL ./micro-suite -k skipalltests], [], [ignore],
[ignore])
+])
+
+
## ----------------- ##
## Debugging a test. ##
## ----------------- ##