[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH 05/39] mtest2make: unify tests that appear in multiple suites
From: |
Paolo Bonzini |
Subject: |
[PATCH 05/39] mtest2make: unify tests that appear in multiple suites |
Date: |
Wed, 2 Sep 2020 08:58:43 -0400 |
Whenever a test appears in multiple suites, the rules generated
by mtest2make are currently running it twice. Instead, after
this patch we generate a phony target for each test and we have
a generic "run-tests" target depend on all the tests that were
chosen on the command line. Tests that appear in multiple suites
will be added to the prerequisites just once.
This has other advantages: it removes the handling of -k and
it increases parallelism.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
scripts/mtest2make.py | 69 ++++++++++++++++++++++---------------------
1 file changed, 35 insertions(+), 34 deletions(-)
diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 27425080cf..e5dcb66bf7 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -24,33 +24,20 @@ SPEED = quick
.test-human-exitcode = $1 $(PYTHON) scripts/test-driver.py $(if $4,-C$4) $(if
$(V),--verbose) -- $2 < /dev/null
.test-tap-tap = $1 $(if $4,(cd $4 && $2),$2) < /dev/null | sed "s/^[a-z][a-z]*
[0-9]*/& $3/" || true
.test-tap-exitcode = printf "%s\\n" 1..1 "`$1 $(if $4,(cd $4 && $2),$2) <
/dev/null > /dev/null || echo "not "`ok 1 $3"
-.test.print = echo $(if $(V),'$1 $2','Running test $3') >&3
+.test.human-print = echo $(if $(V),'$1 $2','Running test $3') &&
.test.env = MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$(( $${RANDOM:-0} % 255 + 1))}
# $1 = test name, $2 = test target (human or tap)
-.test.run = $(call
.test.print,$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1)) && $(call
.test-$2-$(.test.driver.$1),$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1),$(.test.dir.$1))
+.test.run = $(call
.test.$2-print,$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1)) $(call
.test-$2-$(.test.driver.$1),$(.test.env.$1),$(.test.cmd.$1),$(.test.name.$1),$(.test.dir.$1))
-define .test.human_k
- @exec 3>&1; rc=0; $(foreach TEST, $1, $(call .test.run,$(TEST),human)
|| rc=$$?;) \\
- exit $$rc
-endef
-define .test.human_no_k
- $(foreach TEST, $1, @exec 3>&1; $(call .test.run,$(TEST),human)
-)
-endef
-.test.human = \\
- $(if $(findstring k, $(MAKEFLAGS)), $(.test.human_k),
$(.test.human_no_k))
-
-define .test.tap
- @exec 3>&1; { $(foreach TEST, $1, $(call .test.run,$(TEST),tap); ) } \\
- | ./scripts/tap-merge.pl | tee "$@" \\
- | ./scripts/tap-driver.pl $(if $(V),, --show-failures-only)
-endef
+.test.output-format = human
''')
-suites = defaultdict(Suite)
+introspect = json.load(sys.stdin)
i = 0
-for test in json.load(sys.stdin):
+
+def process_tests(test, suites):
+ global i
env = ' '.join(('%s=%s' % (shlex.quote(k), shlex.quote(v))
for k, v in test['env'].items()))
executable = test['cmd'][0]
@@ -89,22 +76,36 @@ for test in json.load(sys.stdin):
suites[s].tests.append(i)
suites[s].executables.add(executable)
-print('.PHONY: check check-report.tap')
-print('check:')
-print('check-report.tap:')
-print('\t@cat $^ | scripts/tap-merge.pl >$@')
-for name, suite in suites.items():
+def emit_prolog(suites, prefix):
+ all_tap = ' '.join(('%s-report-%s.tap' % (prefix, k) for k in
suites.keys()))
+ print('.PHONY: %s %s-report.tap %s' % (prefix, prefix, all_tap))
+ print('%s: run-tests' % (prefix,))
+ print('%s-report.tap %s: %s-report%%.tap:' % (prefix, all_tap, prefix))
+ print('''\t$(MAKE) .test.output-format=tap -Otarget check$* |
./scripts/tap-merge.pl | tee "$@" \\
+ | ./scripts/tap-driver.pl $(if $(V),, --show-failures-only)''')
+
+def emit_suite(name, suite, prefix):
executables = ' '.join(suite.executables)
slow_test_numbers = ' '.join((str(x) for x in suite.slow_tests))
test_numbers = ' '.join((str(x) for x in suite.tests))
print('.test.suite-quick.%s := %s' % (name, test_numbers))
print('.test.suite-slow.%s := $(.test.suite-quick.%s) %s' % (name, name,
slow_test_numbers))
- print('check-build: %s' % executables)
- print('.PHONY: check-%s' % name)
- print('.PHONY: check-report-%s.tap' % name)
- print('check: check-%s' % name)
- print('check-%s: all %s' % (name, executables))
- print('\t$(call .test.human, $(.test.suite-$(SPEED).%s))' % (name, ))
- print('check-report.tap: check-report-%s.tap' % name)
- print('check-report-%s.tap: %s' % (name, executables))
- print('\t$(call .test.tap, $(.test.suite-$(SPEED).%s))' % (name, ))
+ print('%s-build: %s' % (prefix, executables))
+ print('.PHONY: %s-%s' % (prefix, name))
+ print('.PHONY: %s-report-%s.tap' % (prefix, name))
+ print('%s-%s: run-tests' % (prefix, name))
+ print('ifneq ($(filter check check-%s, $(MAKECMDGOALS)),)' % name)
+ print('.tests += $(.test.suite-$(SPEED).%s)' % name)
+ print('endif')
+
+testsuites = defaultdict(Suite)
+for test in introspect:
+ process_tests(test, testsuites)
+emit_prolog(testsuites, 'check')
+for name, suite in testsuites.items():
+ emit_suite(name, suite, 'check')
+
+print('.PHONY: $(patsubst %, run-test-%, $(.tests))')
+print('$(patsubst %, run-test-%, $(sort $(.tests))): run-test-%: all')
+print('\t@$(call .test.run,$*,$(.test.output-format))')
+print('run-tests: $(patsubst %, run-test-%, $(.tests))')
--
2.26.2
- [PATCH 00/39] Next round of Meson fixes and cleanups, Paolo Bonzini, 2020/09/02
- [PATCH 01/39] configure: Add system = 'linux' for meson when cross-compiling, Paolo Bonzini, 2020/09/02
- [PATCH 02/39] mtest2make: split environment from test command, Paolo Bonzini, 2020/09/02
- [PATCH 03/39] mtest2make: split working directory from test command, Paolo Bonzini, 2020/09/02
- [PATCH 07/39] configure: do not include absolute paths in -I and -L paths, Paolo Bonzini, 2020/09/02
- [PATCH 14/39] meson: convert qht-bench, Paolo Bonzini, 2020/09/02
- [PATCH 06/39] meson: remove b_lundef option, Paolo Bonzini, 2020/09/02
- [PATCH 11/39] meson: build qapi tests library, Paolo Bonzini, 2020/09/02
- [PATCH 08/39] configure: include cross sdl2-config in meson cross file, Paolo Bonzini, 2020/09/02
- [PATCH 05/39] mtest2make: unify tests that appear in multiple suites,
Paolo Bonzini <=
- [PATCH 15/39] tests: qga has virtio-serial by default when host has it, Paolo Bonzini, 2020/09/02
- [PATCH 12/39] meson: declare tasn1 dependency, Paolo Bonzini, 2020/09/02
- [PATCH 10/39] meson: fix libqos linking, Paolo Bonzini, 2020/09/02
- [PATCH 21/39] tests: do not print benchmark output to stdout, Paolo Bonzini, 2020/09/02
- [PATCH 23/39] tests/migration/stress: remove unused exit_success, Paolo Bonzini, 2020/09/02
- [PATCH 24/39] meson: fix migration/stress compilation with glibc>=2.30, Paolo Bonzini, 2020/09/02
- [PATCH 19/39] meson: convert vhost-user-bridge, Paolo Bonzini, 2020/09/02
- [PATCH 20/39] meson: convert atomic*-bench, Paolo Bonzini, 2020/09/02
- [PATCH 13/39] meson: declare keyutils dependency, Paolo Bonzini, 2020/09/02
- [PATCH 09/39] ninjatool: use constant names for stamp files, Paolo Bonzini, 2020/09/02