[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v4 21/40] gitlab: build all container images during CI
From: |
Alex Bennée |
Subject: |
[PATCH v4 21/40] gitlab: build all container images during CI |
Date: |
Wed, 1 Jul 2020 14:56:33 +0100 |
From: Daniel P. Berrangé <berrange@redhat.com>
We have a number of container images in tests/docker/dockerfiles
that are intended to provide well defined environments for doing
test builds. We want our CI system to use these containers too.
This introduces builds of all of them as the first stage in the
CI, so that the built containers are available for later build
jobs. The containers are setup to use the GitLab container
registry as the cache, so we only pay the penalty of the full
build when the dockerfiles change. The main qemu-project/qemu
repo is used as a second cache, so that users forking QEMU will
see a fast turnaround time on their CI jobs.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200622153318.751107-3-berrange@redhat.com>
[AJB: tweak the tag format]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
---
v4
- rm leading and trailing ws
---
.gitlab-ci.d/containers.yml | 245 ++++++++++++++++++++++++++++++++++++
.gitlab-ci.yml | 3 +
2 files changed, 248 insertions(+)
create mode 100644 .gitlab-ci.d/containers.yml
diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
new file mode 100644
index 00000000000..2afd01f1d17
--- /dev/null
+++ b/.gitlab-ci.d/containers.yml
@@ -0,0 +1,245 @@
+.container_job_template: &container_job_definition
+ image: docker:stable
+ stage: containers
+ services:
+ - docker:dind
+ before_script:
+ - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
+ - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
+ - docker info
+ - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p
"$CI_REGISTRY_PASSWORD"
+ script:
+ - docker pull "$TAG" || docker pull "$COMMON_TAG" || true
+ - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/,"
tests/docker/dockerfiles/$NAME.docker
+ - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG"
-f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles
+ - docker push "$TAG"
+ after_script:
+ - docker logout
+
+amd64-centos7-container:
+ <<: *container_job_definition
+ variables:
+ NAME: centos7
+
+amd64-centos8-container:
+ <<: *container_job_definition
+ variables:
+ NAME: centos8
+
+amd64-debian10-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian10
+
+amd64-debian11-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian11
+
+amd64-debian9-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian9
+
+amd64-debian9-mxe-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian9-container']
+ variables:
+ NAME: debian9-mxe
+
+alpha-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-alpha-cross
+
+amd64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-amd64-cross
+
+amd64-debian-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-amd64
+
+arm64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-arm64-cross
+
+arm64-test-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian11-container']
+ variables:
+ NAME: debian-arm64-test-cross
+
+armel-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-armel-cross
+
+armhf-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-armhf-cross
+
+hppa-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-hppa-cross
+
+m68k-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-m68k-cross
+
+mips64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-mips64-cross
+
+mips64el-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-mips64el-cross
+
+mips-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-mips-cross
+
+mipsel-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-mipsel-cross
+
+powerpc-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-powerpc-cross
+
+ppc64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-ppc64-cross
+
+ppc64el-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-ppc64el-cross
+
+riscv64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-riscv64-cross
+
+s390x-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-s390x-cross
+
+sh4-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-sh4-cross
+
+sparc64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
+ variables:
+ NAME: debian-sparc64-cross
+
+tricore-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer2
+ needs: ['amd64-debian9-container']
+ variables:
+ NAME: debian-tricore-cross
+
+win32-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer3
+ needs: ['amd64-debian9-mxe-container']
+ variables:
+ NAME: debian-win32-cross
+
+win64-debian-cross-container:
+ <<: *container_job_definition
+ stage: containers-layer3
+ needs: ['amd64-debian9-mxe-container']
+ variables:
+ NAME: debian-win64-cross
+
+xtensa-debian-cross-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-xtensa-cross
+
+cris-fedora-cross-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-cris-cross
+
+amd64-fedora-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora
+
+i386-fedora-cross-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-i386-cross
+
+amd64-ubuntu1804-container:
+ <<: *container_job_definition
+ variables:
+ NAME: ubuntu1804
+
+amd64-ubuntu2004-container:
+ <<: *container_job_definition
+ variables:
+ NAME: ubuntu2004
+
+amd64-ubuntu-container:
+ <<: *container_job_definition
+ variables:
+ NAME: ubuntu
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 9fdc752ea63..72d688875fd 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,10 +1,13 @@
stages:
- containers
+ - containers-layer2
+ - containers-layer3
- build
include:
- local: '/.gitlab-ci.d/edk2.yml'
- local: '/.gitlab-ci.d/opensbi.yml'
+ - local: '/.gitlab-ci.d/containers.yml'
.update_apt_template: &before_script_apt
before_script:
--
2.20.1
- [PATCH v4 10/40] tests/vm: Added a new script for centos.aarch64., (continued)
- [PATCH v4 10/40] tests/vm: Added a new script for centos.aarch64., Alex Bennée, 2020/07/01
- [PATCH v4 11/40] tests/vm: change scripts to use self._config, Alex Bennée, 2020/07/01
- [PATCH v4 07/40] tests/vm: Added configuration file support, Alex Bennée, 2020/07/01
- [PATCH v4 09/40] tests/vm: Added a new script for ubuntu.aarch64., Alex Bennée, 2020/07/01
- [PATCH v4 08/40] tests/vm: Add common Ubuntu python module, Alex Bennée, 2020/07/01
- [PATCH v4 05/40] tests/vm: pass args through to BaseVM's __init__, Alex Bennée, 2020/07/01
- [PATCH v4 25/40] tests/docker: add packages needed for check-acceptance, Alex Bennée, 2020/07/01
- [PATCH v4 06/40] tests/vm: Add configuration to basevm.py, Alex Bennée, 2020/07/01
- [PATCH v4 24/40] tests/docker: add --registry support to tooling, Alex Bennée, 2020/07/01
- [PATCH v4 21/40] gitlab: build all container images during CI,
Alex Bennée <=
- [PATCH v4 20/40] gitlab: introduce explicit "container" and "build" stages, Alex Bennée, 2020/07/01
- [PATCH v4 19/40] gitlab-ci: Fix the change rules after moving the YML files, Alex Bennée, 2020/07/01
- [PATCH v4 22/40] gitlab: convert jobs to use custom built containers, Alex Bennée, 2020/07/01
- [PATCH v4 23/40] gitlab: build containers with buildkit and metadata, Alex Bennée, 2020/07/01
- [PATCH v4 37/40] gitlab: limit re-builds of the containers, Alex Bennée, 2020/07/01
- [PATCH v4 26/40] tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab, Alex Bennée, 2020/07/01
- [PATCH v4 34/40] gitlab: enable check-tcg for linux-user tests, Alex Bennée, 2020/07/01
- [PATCH v4 15/40] tests/vm: allow us to take advantage of MTTCG, Alex Bennée, 2020/07/01
- [PATCH v4 35/40] gitlab: add avocado asset caching, Alex Bennée, 2020/07/01
- [PATCH v4 16/40] tests/docker: check for an parameters not empty string, Alex Bennée, 2020/07/01