Re: [PATCH-for-5.2 v3] tests/acceptance: Disable Spartan-3A DSP 1800A test

[Thomas Huth]

On 09/11/2020 10.17, Philippe Mathieu-Daudé wrote:
> This test is regularly failing on CI:
> 
>    (05/34) 
> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
>   Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 
> 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
>   ...
>   Freeing unused kernel memory: 1444K
>   This architecture does not have kernel memory protection.
>   [nothing happens here]
>   Runner error occurred: Timeout reached (90.91 s)
> 
> This is a regression. Until someone figure out the problem,
> disable the test to keep CI pipeline useful.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> v3: Simply disable with @skip
> ---
>  tests/acceptance/boot_linux_console.py | 2 ++
>  tests/acceptance/replay_kernel.py      | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py 
> b/tests/acceptance/boot_linux_console.py
> index 8f433a67f84..cc6ec0f8c15 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -13,6 +13,7 @@
>  import gzip
>  import shutil
>  
> +from avocado import skip
>  from avocado import skipUnless
>  from avocado_qemu import Test
>  from avocado_qemu import exec_command_and_wait_for_pattern
> @@ -1025,6 +1026,7 @@ def test_m68k_mcf5208evb(self):
>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> diff --git a/tests/acceptance/replay_kernel.py 
> b/tests/acceptance/replay_kernel.py
> index 00c228382bd..772633b01da 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -14,6 +14,7 @@
>  import logging
>  import time
>  
> +from avocado import skip
>  from avocado import skipIf
>  from avocado import skipUnless
>  from avocado_qemu import wait_for_console_pattern
> @@ -280,6 +281,7 @@ def test_m68k_mcf5208evb(self):
>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> 

Acked-by: Thomas Huth <thuth@redhat.com>

FWIW, I tried to bisect this issue today to see where it broke, but if I run
the test often enough, I even got failures with QEMU v5.0 already, so it's
likely not a new issue after all:

make check-venv
for ((i=0;i<500;i++)); do echo $i ; \
 AVOCADO_ALLOW_UNTRUSTED_CODE=1 tests/venv/bin/avocado run \
 
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800
 \
 || break ; sleep 0.1 ; \
done