[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 07/46] mtest2make: unify tests that appear in multiple suites
From: |
Paolo Bonzini |
Subject: |
[PULL 07/46] mtest2make: unify tests that appear in multiple suites |
Date: |
Fri, 4 Sep 2020 07:40:43 -0400 |
Whenever a test appears in multiple suites, the rules generated
by mtest2make are currently running it twice. Instead, after
this patch we generate a phony target for each test and we have
a generic "run-tests" target depend on all the tests that were
chosen on the command line. Tests that appear in multiple suites
will be added to the prerequisites just once.
This has other advantages: it removes the handling of -k and
it increases parallelism.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
scripts/mtest2make.py | 74 ++++++++++++++++++++++---------------------
1 file changed, 38 insertions(+), 36 deletions(-)
diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 27425080cf..af6dd0d3b1 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -24,33 +24,20 @@ SPEED = quick
.test-human-exitcode = $1 $(PYTHON) scripts/test-driver.py $(if $4,-C$4) $(if
$(V),--verbose) -- $2 < /dev/null
.test-tap-tap = $1 $(if $4,(cd $4 && $2),$2) < /dev/null | sed "s/^[a-z][a-z]*
[0-9]*/& $3/" || true
.test-tap-exitcode = printf "%s\\n" 1..1 "`$1 $(if $4,(cd $4 && $2),$2) <
/dev/null > /dev/null || echo "not "`ok 1 $3"
-.test.print = echo $(if $(V),'$1 $2','Running test $3') >&3
+.test.human-print = echo $(if $(V),'$1 $2','Running test $3') &&
.test.env = MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$(( $${RANDOM:-0} % 255 + 1))}
# $1 = test name, $2 = test target (human or tap)
-.test.run = $(call
.test.print,$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1)) && $(call
.test-$2-$(.test.driver.$1),$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1),$(.test.dir.$1))
+.test.run = $(call
.test.$2-print,$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1)) $(call
.test-$2-$(.test.driver.$1),$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1),$(.test.dir.$1))
-define .test.human_k
- @exec 3>&1; rc=0; $(foreach TEST, $1, $(call .test.run,$(TEST),human)
|| rc=$$?;) \\
- exit $$rc
-endef
-define .test.human_no_k
- $(foreach TEST, $1, @exec 3>&1; $(call .test.run,$(TEST),human)
-)
-endef
-.test.human = \\
- $(if $(findstring k, $(MAKEFLAGS)), $(.test.human_k),
$(.test.human_no_k))
-
-define .test.tap
- @exec 3>&1; { $(foreach TEST, $1, $(call .test.run,$(TEST),tap); ) } \\
- | ./scripts/tap-merge.pl | tee "$@" \\
- | ./scripts/tap-driver.pl $(if $(V),, --show-failures-only)
-endef
+.test.output-format = human
''')
-suites = defaultdict(Suite)
+introspect = json.load(sys.stdin)
i = 0
-for test in json.load(sys.stdin):
+
+def process_tests(test, suites):
+ global i
env = ' '.join(('%s=%s' % (shlex.quote(k), shlex.quote(v))
for k, v in test['env'].items()))
executable = test['cmd'][0]
@@ -75,6 +62,9 @@ for test in json.load(sys.stdin):
print('.test.driver.%d := %s' % (i, driver))
print('.test.env.%d := $(.test.env) %s' % (i, env))
print('.test.cmd.%d := %s' % (i, cmd))
+ print('.PHONY: run-test-%d' % (i,))
+ print('run-test-%d: all' % (i,))
+ print('\t@$(call .test.run,%d,$(.test.output-format))' % (i,))
test_suites = test['suite'] or ['default']
is_slow = any(s.endswith('-slow') for s in test_suites)
@@ -89,22 +79,34 @@ for test in json.load(sys.stdin):
suites[s].tests.append(i)
suites[s].executables.add(executable)
-print('.PHONY: check check-report.tap')
-print('check:')
-print('check-report.tap:')
-print('\t@cat $^ | scripts/tap-merge.pl >$@')
-for name, suite in suites.items():
+def emit_prolog(suites, prefix):
+ all_tap = ' '.join(('%s-report-%s.tap' % (prefix, k) for k in
suites.keys()))
+ print('.PHONY: %s %s-report.tap %s' % (prefix, prefix, all_tap))
+ print('%s: run-tests' % (prefix,))
+ print('%s-report.tap %s: %s-report%%.tap: all' % (prefix, all_tap, prefix))
+ print('''\t$(MAKE) .test.output-format=tap --quiet -Otarget V=1 %s$* |
./scripts/tap-merge.pl | tee "$@" \\
+ | ./scripts/tap-driver.pl $(if $(V),, --show-failures-only)''' %
(prefix, ))
+
+def emit_suite(name, suite, prefix):
executables = ' '.join(suite.executables)
slow_test_numbers = ' '.join((str(x) for x in suite.slow_tests))
test_numbers = ' '.join((str(x) for x in suite.tests))
- print('.test.suite-quick.%s := %s' % (name, test_numbers))
- print('.test.suite-slow.%s := $(.test.suite-quick.%s) %s' % (name, name,
slow_test_numbers))
- print('check-build: %s' % executables)
- print('.PHONY: check-%s' % name)
- print('.PHONY: check-report-%s.tap' % name)
- print('check: check-%s' % name)
- print('check-%s: all %s' % (name, executables))
- print('\t$(call .test.human, $(.test.suite-$(SPEED).%s))' % (name, ))
- print('check-report.tap: check-report-%s.tap' % name)
- print('check-report-%s.tap: %s' % (name, executables))
- print('\t$(call .test.tap, $(.test.suite-$(SPEED).%s))' % (name, ))
+ target = '%s-%s' % (prefix, name)
+ print('.test.quick.%s := %s' % (target, test_numbers))
+ print('.test.slow.%s := $(.test.quick.%s) %s' % (target, target,
slow_test_numbers))
+ print('%s-build: %s' % (prefix, executables))
+ print('.PHONY: %s' % (target, ))
+ print('.PHONY: %s-report-%s.tap' % (prefix, name))
+ print('%s: run-tests' % (target, ))
+ print('ifneq ($(filter %s %s, $(MAKECMDGOALS)),)' % (target, prefix))
+ print('.tests += $(.test.$(SPEED).%s)' % (target, ))
+ print('endif')
+
+testsuites = defaultdict(Suite)
+for test in introspect:
+ process_tests(test, testsuites)
+emit_prolog(testsuites, 'check')
+for name, suite in testsuites.items():
+ emit_suite(name, suite, 'check')
+
+print('run-tests: $(patsubst %, run-test-%, $(.tests))')
--
2.26.2
- [PULL 00/46] Next round of Meson bugfixes and cleanups, Paolo Bonzini, 2020/09/04
- [PULL 02/46] tests/Makefile: test-image-locking needs CONFIG_POSIX, Paolo Bonzini, 2020/09/04
- [PULL 01/46] qemu-iotests: move check-block back to Makefiles, Paolo Bonzini, 2020/09/04
- [PULL 08/46] meson: remove b_lundef option, Paolo Bonzini, 2020/09/04
- [PULL 06/46] mtest2make: hide output of successful tests, Paolo Bonzini, 2020/09/04
- [PULL 04/46] mtest2make: split environment from test command, Paolo Bonzini, 2020/09/04
- [PULL 09/46] configure: do not include absolute paths in -I and -L paths, Paolo Bonzini, 2020/09/04
- [PULL 16/46] meson: convert qht-bench, Paolo Bonzini, 2020/09/04
- [PULL 14/46] meson: declare tasn1 dependency, Paolo Bonzini, 2020/09/04
- [PULL 07/46] mtest2make: unify tests that appear in multiple suites,
Paolo Bonzini <=
- [PULL 13/46] meson: build qapi tests library, Paolo Bonzini, 2020/09/04
- [PULL 12/46] meson: fix libqos linking, Paolo Bonzini, 2020/09/04
- [PULL 10/46] configure: include cross sdl2-config in meson cross file, Paolo Bonzini, 2020/09/04
- [PULL 17/46] meson: convert the unit tests, Paolo Bonzini, 2020/09/04
- [PULL 20/46] meson: convert vhost-user-bridge, Paolo Bonzini, 2020/09/04
- [PULL 15/46] meson: declare keyutils dependency, Paolo Bonzini, 2020/09/04
- [PULL 05/46] mtest2make: split working directory from test command, Paolo Bonzini, 2020/09/04
- [PULL 03/46] tests: handling signal on win32 properly, Paolo Bonzini, 2020/09/04
- [PULL 11/46] ninjatool: use constant names for stamp files, Paolo Bonzini, 2020/09/04
- [PULL 18/46] meson: move keyutils dependency check, Paolo Bonzini, 2020/09/04