[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2] tests/acceptance: allow control over tags during check-ac

From: Wainer dos Santos Moschetta
Subject: Re: [PATCH v2] tests/acceptance: allow control over tags during check-acceptance
Date: Fri, 9 Jul 2021 16:32:28 -0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1


On 7/7/21 12:24 PM, Willian Rampazzo wrote:
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.
I always wanted that feature, thanks for implementing it!

This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.

This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.


AVOCADO_TAGS="foo bar baz" make check-acceptance

If I run `AVOCADO_TAGS="machine:malta arch:mips" make check-acceptance` it is going to get the tests with 'machine:malta' *or* 'arch:mips' tags, whereas with AVOCADO_TAGS="machine:malta,arch:mips" it gets those with 'machine:malta' *and* 'arch:mips', i.e., the semantic is different. I think it deserves a mention in the docs/devel/testing.rst, maybe linking to Avocado's doc [1] as well.

[1] https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
  docs/devel/testing.rst |  8 ++++++++
  tests/Makefile.include | 12 +++++++++---
  2 files changed, 17 insertions(+), 3 deletions(-)

Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4e42392810..f6e4fc08a7 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -760,6 +760,14 @@ in the current directory, tagged as "quick", run:
avocado run -t quick . +It is also possible to run tests based on tags using the
+``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment
+.. code::
+   AVOCADO_TAGS=quick make check-acceptance
  The ``avocado_qemu.Test`` base test class
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8f220e15d1..d464f06e4a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
  # Any number of command separated loggers are accepted.  For more
  # information please refer to "avocado --help".
-AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
+       AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
+                                                $(filter %-softmmu,$(TARGETS)))
+       AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
        $(call quiet-command, \
@@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) 
        $(call quiet-command, \
              $(TESTS_VENV_DIR)/bin/python -m avocado \
              --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) 
-            --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
-            $(AVOCADO_TAGS) \
+            $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
+                       --filter-by-tags-include-empty-key) \
+            $(AVOCADO_CMDLINE_TAGS) \
              $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
              "AVOCADO", "tests/acceptance")

reply via email to

[Prev in Thread] Current Thread [Next in Thread]