Re: [PATCH] gitlab: remove unreliable avocado CI jobs

[Peter Maydell]

On Tue, 12 Sept 2023 at 21:00, Thomas Huth <thuth@redhat.com> wrote:
> Please don't remove the whole job! Just disable the failing tests within the 
> job, e.g.:
>
> diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
> --- a/tests/avocado/replay_kernel.py
> +++ b/tests/avocado/replay_kernel.py
> @@ -503,6 +503,7 @@ def do_test_mips_malta32el_nanomips(self, kernel_path_xz):
>           console_pattern = 'Kernel command line: %s' % kernel_command_line
>           self.run_rr(kernel_path, kernel_command_line, console_pattern, 
> shift=5)
>
> +    @skipIf(os.getenv('GITLAB_CI'), 'Skipping unstable test on GitLab')
>       def test_mips_malta32el_nanomips_4k(self):
>           """
>           :avocado: tags=arch:mipsel

Please don't skip unstable tests on gitlab only. If they're
unstable, then nobody wants to be running them and wondering
if these are flaky tests or real issues, whether theyr'e doing
it on gitlab or locally. (I know we already have a lot of these,
but the effect is that instead of saying 'make check-avocado'
you have to say 'GITLAB_CI=1 make check-avocado'.)

thanks
-- PMM