[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-devel] [PATCH 2/2] Acceptance tests: add make rule for running the

From: Cleber Rosa
Subject: [Qemu-devel] [PATCH 2/2] Acceptance tests: add make rule for running them
Date: Thu, 20 Sep 2018 11:19:56 -0400

The acceptance (aka functional, aka Avocado-based) tests are
Python files located in "tests/acceptance" that need to be run
with the Avocado libs and test runner.

Let's provide a convenient way for QEMU developers to run them,
by making use of the tests-venv with the required setup.

Also, while the Avocado test runner will take care of creating a
location to save test results to, it was understood that it's better
if the results are kept within the build tree.

Signed-off-by: Cleber Rosa <address@hidden>
 docs/devel/testing.rst      | 28 +++++++++++++++++++++++-----
 tests/Makefile.include      | 17 +++++++++++++++--
 tests/venv-requirements.txt |  1 +
 3 files changed, 39 insertions(+), 7 deletions(-)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 727c4019b5..0fbf0d0aac 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -545,10 +545,24 @@ Tests based on ``avocado_qemu.Test`` can easily:
+Running tests
-To install Avocado and its dependencies, run:
+You can run the acceptance tests simply by executing:
+.. code::
+  make check-acceptance
+This involves the automatic creation of Python virtual environment
+within the build tree (at ``tests/venv``) which will have all the
+right dependencies, and will save tests results also within the
+build tree (at ``tests/results``).
+Manual Installation
+To manually install Avocado and its dependencies, run:
 .. code::
@@ -689,11 +703,15 @@ The exact QEMU binary to be used on QEMUMachine.
 Uninstalling Avocado
-If you've followed the installation instructions above, you can easily
-uninstall Avocado.  Start by listing the packages you have installed::
+If you've followed the manual installation instructions above, you can
+easily uninstall Avocado.  Start by listing the packages you have
   pip list --user
 And remove any package you want with::
   pip uninstall <package_name>
+If you've used ``make check-acceptance``, the Python virtual environment where
+Avocado is installed will be cleaned up as part of ``make check-clean``.
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 9bb90a83d4..8cef694954 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -11,6 +11,7 @@ check-help:
        @echo " $(MAKE) check-qapi-schema    Run QAPI schema tests"
        @echo " $(MAKE) check-block          Run block tests"
        @echo " $(MAKE) check-tcg            Run TCG tests"
+       @echo " $(MAKE) check-acceptance     Run all acceptance (functional) 
        @echo " $(MAKE) check-report.html    Generates an HTML test report"
        @echo " $(MAKE) check-venv           Creates a Python venv for tests"
        @echo " $(MAKE) check-clean          Clean the tests"
@@ -1002,10 +1003,11 @@ check-decodetree:
 # Python venv for running tests
-.PHONY: check-venv
+.PHONY: check-venv check-acceptance
        $(call quiet-command, \
@@ -1015,8 +1017,19 @@ $(TESTS_VENV_DIR):
             $(TESTS_VENV_DIR)/bin/pip -q install -r $(TESTS_VENV_REQ), \
             PIP, $(TESTS_VENV_REQ))
+       $(call quiet-command, mkdir -p $@, \
+            MKDIR, $@)
 check-venv: $(TESTS_VENV_DIR)
+check-acceptance: check-venv $(TESTS_RESULTS_DIR)
+       $(call quiet-command, \
+            $(TESTS_VENV_DIR)/bin/avocado \
+            --show=none run --job-results-dir=$(TESTS_RESULTS_DIR) 
--failfast=on \
+            $(SRC_PATH)/tests/acceptance, \
+            "AVOCADO", "tests/acceptance")
 # Consolidated targets
 .PHONY: check-qapi-schema check-qtest check-unit check check-clean
@@ -1030,7 +1043,7 @@ check-clean:
        rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
        rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), 
$(check-qtest-$(target)-y)) $(check-qtest-generic-y))
        rm -f tests/test-qapi-gen-timestamp
-       rm -rf $(TESTS_VENV_DIR)
 clean: check-clean
diff --git a/tests/venv-requirements.txt b/tests/venv-requirements.txt
index d39f9d1576..1734d0ce27 100644
--- a/tests/venv-requirements.txt
+++ b/tests/venv-requirements.txt
@@ -1,3 +1,4 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1

reply via email to

[Prev in Thread] Current Thread [Next in Thread]