Re: [PATCH v4 28/40] tests/acceptance: skip multicore mips_malta tests on GitLab

[Alex Bennée]

Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> writes:

> On Wed, Jul 1, 2020 at 4:03 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> For some reason these tests fail all the time on GitLab. I can
>> re-create the hang around 3% of the time locally but it doesn't seem
>> to be MTTCG related. For now skipIf on GITLAB_CI.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Cc: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
>> ---
>
> Alex,
>
> Thanks for having this test at all. I will review its content, but
> here is my stupid question:
>
> How can I, as a regular developer, repro the test in question? I am
> not familiar with GitLab at all.

To run the acceptance tests you just run:

  make check-acceptance

To run the individual test:

  ./tests/venv/bin/avocado run 
tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i64
00_framebuffer_logo_8cores

I used my retry.py script to just loop running the test for 100 times to
calculate the failure rate.

>
> Thanks,
> Aleksandar
>
>>  tests/acceptance/machine_mips_malta.py | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/tests/acceptance/machine_mips_malta.py 
>> b/tests/acceptance/machine_mips_malta.py
>> index 92b4f28a112..7c9a4ee4d2d 100644
>> --- a/tests/acceptance/machine_mips_malta.py
>> +++ b/tests/acceptance/machine_mips_malta.py
>> @@ -15,6 +15,7 @@ from avocado import skipUnless
>>  from avocado_qemu import Test
>>  from avocado_qemu import wait_for_console_pattern
>>  from avocado.utils import archive
>> +from avocado import skipIf
>>
>>
>>  NUMPY_AVAILABLE = True
>> @@ -99,6 +100,7 @@ class MaltaMachineFramebuffer(Test):
>>          """
>>          self.do_test_i6400_framebuffer_logo(1)
>>
>> +    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
>>      def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>>          """
>>          :avocado: tags=arch:mips64el
>> @@ -108,6 +110,7 @@ class MaltaMachineFramebuffer(Test):
>>          """
>>          self.do_test_i6400_framebuffer_logo(7)
>>
>> +    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
>>      def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>>          """
>>          :avocado: tags=arch:mips64el
>> --
>> 2.20.1
>>


-- 
Alex Bennée