[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner
From: |
Erik Skultety |
Subject: |
Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook |
Date: |
Mon, 22 Feb 2021 07:36:28 +0100 |
On Fri, Feb 19, 2021 at 04:58:37PM -0500, Cleber Rosa wrote:
> To have the jobs dispatched to custom runners, gitlab-runner must
> be installed, active as a service and properly configured. The
> variables file and playbook introduced here should help with those
> steps.
>
> The playbook introduced here covers a number of different Linux
> distributions and FreeBSD, and are intended to provide a reproducible
> environment.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> docs/devel/ci.rst | 58 ++++++++++++++++++++++++++
> scripts/ci/setup/.gitignore | 1 +
> scripts/ci/setup/gitlab-runner.yml | 65 ++++++++++++++++++++++++++++++
> scripts/ci/setup/vars.yml.template | 13 ++++++
> 4 files changed, 137 insertions(+)
> create mode 100644 scripts/ci/setup/.gitignore
> create mode 100644 scripts/ci/setup/gitlab-runner.yml
> create mode 100644 scripts/ci/setup/vars.yml.template
>
> diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> index a556558435..9f9c4bd3f9 100644
> --- a/docs/devel/ci.rst
> +++ b/docs/devel/ci.rst
> @@ -56,3 +56,61 @@ To run the playbook, execute::
>
> cd scripts/ci/setup
> ansible-playbook -i inventory build-environment.yml
> +
> +gitlab-runner setup and registration
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The gitlab-runner agent needs to be installed on each machine that
> +will run jobs. The association between a machine and a GitLab project
> +happens with a registration token. To find the registration token for
> +your repository/project, navigate on GitLab's web UI to:
I think the word order should be "on GitLab's web UI navigate to:"
> +
> + * Settings (the gears like icon), then
> + * CI/CD, then
> + * Runners, and click on the "Expand" button, then
> + * Under "Set up a specific Runner manually", look for the value under
> + "Use the following registration token during setup"
> +
> +Copy the ``scripts/ci/setup/vars.yml.template`` file to
> +``scripts/ci/setup/vars.yml``. Then, set the
> +``gitlab_runner_registration_token`` variable to the value obtained
> +earlier.
> +
> +.. note:: gitlab-runner is not available from the standard location
> + for all OS and architectures combinations. For some systems,
> + a custom build may be necessary. Some builds are avaiable
s/avaiable/available
> + at https://cleber.fedorapeople.org/gitlab-runner/ and this
> + URI may be used as a value on ``vars.yml``
> +
> +To run the playbook, execute::
> +
> + cd scripts/ci/setup
> + ansible-playbook -i inventory gitlab-runner.yml
> +
> +Following the registration, it's necessary to configure the runner tags,
> +and optionally other configurations on the GitLab UI. Navigate to:
> +
> + * Settings (the gears like icon), then
> + * CI/CD, then
> + * Runners, and click on the "Expand" button, then
> + * "Runners activated for this project", then
> + * Click on the "Edit" icon (next to the "Lock" Icon)
> +
> +Under tags, add values matching the jobs a runner should run. For a
> +Ubuntu 20.04 aarch64 system, the tags should be set as::
> +
> + ubuntu_20.04,aarch64
> +
> +Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
> +would contain::
> +
> + ubuntu-20.04-aarch64-all:
> + tags:
> + - ubuntu_20.04
> + - aarch64
> +
> +It's also recommended to:
> +
> + * increase the "Maximum job timeout" to something like ``2h``
> + * uncheck the "Run untagged jobs" check box
> + * give it a better Description
> diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
> new file mode 100644
> index 0000000000..f112d05dd0
> --- /dev/null
> +++ b/scripts/ci/setup/.gitignore
> @@ -0,0 +1 @@
> +vars.yml
> \ No newline at end of file
> diff --git a/scripts/ci/setup/gitlab-runner.yml
> b/scripts/ci/setup/gitlab-runner.yml
> new file mode 100644
> index 0000000000..ab1944965f
> --- /dev/null
> +++ b/scripts/ci/setup/gitlab-runner.yml
> @@ -0,0 +1,65 @@
> +---
> +- name: Installation of gitlab-runner
> + hosts: all
> + vars_files:
> + - vars.yml
> + tasks:
> + - debug:
> + msg: 'Checking for a valid GitLab registration token'
> + failed_when: "gitlab_runner_registration_token ==
> 'PLEASE_PROVIDE_A_VALID_TOKEN'"
> +
> + - name: Checks the availability of official gitlab-runner builds in the
> archive
> + uri:
> + url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{
> gitlab_runner_version }}/binaries/gitlab-runner-linux-386
> + method: HEAD
> + status_code:
> + - 200
> + - 403
> + register: gitlab_runner_available_archive
> +
> + - name: Update base url
> + set_fact:
> + gitlab_runner_base_url:
> https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version
> }}/binaries/gitlab-runner-
> + when: gitlab_runner_available_archive.status == 200
> + - debug:
> + msg: Base gitlab-runner url is {{ gitlab_runner_base_url }}
> +
> + - name: Create a group for the gitlab-runner service
> + group:
> + name: gitlab-runner
> +
> + - name: Create a user for the gitlab-runner service
> + user:
> + user: gitlab-runner
> + group: gitlab-runner
> + comment: GitLab Runner
> + home: /home/gitlab-runner
> + shell: /bin/bash
Totally unimportant (you may as well ignore this comment), but depending on
how much in sync you want to be with libvirt's playbook, the user:group we
create is gitlab:gitlab.
> +
> + - name: Remove the .bash_logout file when on Ubuntu systems
> + file:
> + path: /home/gitlab-runner/.bash_logout
> + state: absent
> + when: "ansible_facts['distribution'] == 'Ubuntu'"
> +
> + - name: Downloads the matching gitlab-runner
> + get_url:
> + dest: /usr/local/bin/gitlab-runner
> + url: "{{ gitlab_runner_base_url }}{{ gitlab_runner_os }}-{{
> gitlab_runner_arch }}"
> + owner: gitlab-runner
> + group: gitlab-runner
> + mode: u=rwx,g=rwx,o=rx
> +
> + - name: Register the gitlab-runner
> + command: "/usr/local/bin/gitlab-runner register --non-interactive
> --url {{ gitlab_runner_server_url }} --registration-token {{
> gitlab_runner_registration_token }} --executor shell --description '{{
> ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"]
> }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"]
> }})'"
> +
> + - name: Install the gitlab-runner service using its own functionality
> + command: /usr/local/bin/gitlab-runner install --user gitlab-runner
> --working-directory /home/gitlab-runner
I'm pretty sure I pointed this out in previous versions, but according to the
docs ^this won't install the runner on FreeBSD as a service. IIRC the answer
was that FreeBSD is not in the priority distro list at the moment and that it
can always be adjusted further down the road - that is fair, no objection, but
then the commit message says that this playbook is creating a reproducible
environment and covers both Linux and FreeBSD which is not true in its
entirety, so either drop it from the commit message or add a small comment here
that the command would actually only work as expected on Linux.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
> + register: gitlab_runner_install_service_result
> + failed_when: "gitlab_runner_install_service_result.rc != 0 and
> \"already exists\" not in gitlab_runner_install_service_result.stderr"
> +
> + - name: Enable the gitlab-runner service
> + service:
> + name: gitlab-runner
> + state: started
> + enabled: yes
> diff --git a/scripts/ci/setup/vars.yml.template
> b/scripts/ci/setup/vars.yml.template
> new file mode 100644
> index 0000000000..621435d030
> --- /dev/null
> +++ b/scripts/ci/setup/vars.yml.template
> @@ -0,0 +1,13 @@
> +# The version of the gitlab-runner to use
> +gitlab_runner_version: 13.1.1
> +# The base location of gitlab-runner binaries, this will be suffixed by
> $OS-$ARCH
> +gitlab_runner_base_url:
> https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-
> +# The URL of the gitlab server to use, usually https://gitlab.com unless
> you're
> +# using a private GitLab instance
> +gitlab_runner_server_url: https://gitlab.com
> +# Defaults to linux, checks can be used to change this
> +gitlab_runner_os: linux
> +# Defaults to amd64 (x86_64), checks can be used to change this
> +gitlab_runner_arch: amd64
> +# A unique token made available by GitLab to your project for registering
> runners
> +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
> --
> 2.25.4
>
- [PATCH v5 2/4] Jobs based on custom runners: build environment docs and playbook, (continued)
- [PATCH v5 2/4] Jobs based on custom runners: build environment docs and playbook, Cleber Rosa, 2021/02/19
- [PATCH v5 1/4] Jobs based on custom runners: documentation and configuration placeholder, Cleber Rosa, 2021/02/19
- [PATCH v5 4/4] Jobs based on custom runners: add job definitions for QEMU's machines, Cleber Rosa, 2021/02/19
- [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Cleber Rosa, 2021/02/19
- Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook,
Erik Skultety <=
- Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Wainer dos Santos Moschetta, 2021/02/22
- Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Philippe Mathieu-Daudé, 2021/02/23
- Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Philippe Mathieu-Daudé, 2021/02/23
- Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Philippe Mathieu-Daudé, 2021/02/23
- Re: [PATCH v5 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Alex Bennée, 2021/02/23