[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 28/41] gitlab: add acceptance testing to system builds
From: |
Alex Bennée |
Subject: |
[PULL 28/41] gitlab: add acceptance testing to system builds |
Date: |
Tue, 7 Jul 2020 08:08:45 +0100 |
As part of migrating things from Travis to GitLab add the acceptance
tests. To do this:
- rename system1 to system-ubuntu-main
- rename system2 to system-fedora-misc
- split into build/check/acceptance
- remove -j from check stages
- use artifacts to save build stage
- add post acceptance template and use
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org>
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index a7abc55a5c6a..93baf98232d7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,8 +1,12 @@
+# Currently we have two build stages after our containers are built:
+# - build (for traditional build and test or first stage build)
+# - test (for test stages, using build artefacts from a build stage)
stages:
- containers
- containers-layer2
- containers-layer3
- build
+ - test
include:
- local: '/.gitlab-ci.d/edk2.yml'
@@ -24,26 +28,85 @@ include:
../configure --enable-werror $CONFIGURE_ARGS ;
fi
- make -j"$JOBS"
- - make -j"$JOBS" $MAKE_CHECK_ARGS
+ - if test -n "$MAKE_CHECK_ARGS";
+ then
+ make $MAKE_CHECK_ARGS ;
+ fi
+
+.native_test_job_template: &native_test_job_definition
+ stage: test
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ script:
+ - cd build
+ - find . -type f -exec touch {} +
+ - make $MAKE_CHECK_ARGS
+
+.post_acceptance_template: &post_acceptance
+ after_script:
+ - cd build
+ - python3 -c 'import json; r =
json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for
t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat
+ - du -chs $HOME/avocado/data/cache
-build-system1:
+build-system-ubuntu-main:
<<: *native_build_job_definition
variables:
IMAGE: ubuntu2004
TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
lm32-softmmu
moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu
ppc-softmmu
riscv64-softmmu sparc-softmmu
+ artifacts:
+ paths:
+ - build
+
+check-system-ubuntu-main:
+ <<: *native_test_job_definition
+ needs:
+ - job: build-system-ubuntu-main
+ artifacts: true
+ variables:
+ IMAGE: ubuntu2004
MAKE_CHECK_ARGS: check
-build-system2:
+acceptance-system-ubuntu-main:
+ <<: *native_test_job_definition
+ needs:
+ - job: build-system-ubuntu-main
+ artifacts: true
+ variables:
+ IMAGE: ubuntu2004
+ MAKE_CHECK_ARGS: check-acceptance
+ <<: *post_acceptance
+
+build-system-fedora-alt:
<<: *native_build_job_definition
variables:
IMAGE: fedora
TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu
riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu
xtensa-softmmu nios2-softmmu or1k-softmmu
+ artifacts:
+ paths:
+ - build
+
+check-system-fedora-alt:
+ <<: *native_test_job_definition
+ needs:
+ - job: build-system-fedora-alt
+ artifacts: true
+ variables:
+ IMAGE: fedora
MAKE_CHECK_ARGS: check
+acceptance-system-fedora-alt:
+ <<: *native_test_job_definition
+ needs:
+ - job: build-system-fedora-alt
+ artifacts: true
+ variables:
+ IMAGE: fedora
+ MAKE_CHECK_ARGS: check-acceptance
+ <<: *post_acceptance
+
build-disabled:
<<: *native_build_job_definition
variables:
diff --git a/.travis.yml b/.travis.yml
index 74158f741b18..c24dfbe377f3 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -289,29 +289,6 @@ jobs:
python: 3.6
- # Acceptance (Functional) tests
- - name: "GCC check-acceptance"
- dist: bionic
- env:
- - CONFIG="--enable-tools
--target-list=aarch64-softmmu,alpha-softmmu,arm-softmmu,m68k-softmmu,microblaze-softmmu,mips-softmmu,mips64el-softmmu,nios2-softmmu,or1k-softmmu,ppc-softmmu,ppc64-softmmu,s390x-softmmu,sh4-softmmu,sparc-softmmu,x86_64-softmmu,xtensa-softmmu"
- - TEST_CMD="make check-acceptance"
- - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-acceptance"
- after_script:
- - python3 -c 'import json; r =
json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for
t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat
- - du -chs $HOME/avocado/data/cache
- addons:
- apt:
- packages:
- - python3-pil
- - python3-pip
- - python3-numpy
- - python3-opencv
- - python3-venv
- - rpm2cpio
- - tesseract-ocr
- - tesseract-ocr-eng
-
-
# Using newer GCC with sanitizers
- name: "GCC9 with sanitizers (softmmu)"
addons:
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 3f4448a20bc7..c316e0d66475 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -944,7 +944,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
get-vm-images
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
--filter-by-tags-include-empty --filter-by-tags-include-empty-key \
$(AVOCADO_TAGS) \
- --failfast=on tests/acceptance, \
+ $(if $(GITLAB_CI),,--failfast=on) tests/acceptance, \
"AVOCADO", "tests/acceptance")
# Consolidated targets
diff --git a/tests/acceptance/replay_kernel.py
b/tests/acceptance/replay_kernel.py
index 60621417ddf0..62d2db8c64be 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -73,7 +73,7 @@ class ReplayKernel(LinuxKernelTest):
logger = logging.getLogger('replay')
logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
- @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
+ @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
def test_x86_64_pc(self):
"""
:avocado: tags=arch:x86_64
--
2.20.1
- Re: [PULL 10/41] python/qemu: Add ConsoleSocket for optional use in QEMUMachine, (continued)
[PULL 12/41] tests/vm: switch from optsparse to argparse, Alex Bennée, 2020/07/07
[PULL 14/41] tests/docker: check for an parameters not empty string, Alex Bennée, 2020/07/07
[PULL 15/41] tests/docker: change tag naming scheme of our images, Alex Bennée, 2020/07/07
[PULL 11/41] tests/vm: Add workaround to consume console, Alex Bennée, 2020/07/07
[PULL 13/41] tests/vm: allow us to take advantage of MTTCG, Alex Bennée, 2020/07/07
[PULL 29/41] tests/tcg: add more default compilers to configure.sh, Alex Bennée, 2020/07/07
[PULL 41/41] tests/qht-bench: Adjust threshold computation, Alex Bennée, 2020/07/07
[PULL 28/41] gitlab: add acceptance testing to system builds,
Alex Bennée <=
[PULL 31/41] linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va, Alex Bennée, 2020/07/07
[PULL 20/41] gitlab: convert jobs to use custom built containers, Alex Bennée, 2020/07/07
[PULL 39/41] travis.yml: Test also the other targets on s390x, Alex Bennée, 2020/07/07
[PULL 18/41] gitlab: introduce explicit "container" and "build" stages, Alex Bennée, 2020/07/07
[PULL 16/41] .gitignore: un-ignore .gitlab-ci.d, Alex Bennée, 2020/07/07
[PULL 17/41] gitlab-ci: Fix the change rules after moving the YML files, Alex Bennée, 2020/07/07
[PULL 27/41] tests/acceptance: skip LinuxInitrd 2gib with v4.16 on GitLab, Alex Bennée, 2020/07/07
[PULL 36/41] containers.yml: build with docker.py tooling, Alex Bennée, 2020/07/07
[PULL 25/41] tests/acceptance: fix dtb path for machine_rx_gdbsim, Alex Bennée, 2020/07/07
[PULL 34/41] gitlab: split build-disabled into two phases, Alex Bennée, 2020/07/07