Re: [PATCH 1/4] Acceptance tests: use an available kernel image package for arm

[Philippe Mathieu-Daudé]

[Cc'ing Daniel who usually have good ideas for that
kind if project-wide problem]

On 9/7/20 6:19 AM, Cleber Rosa wrote:
> Which means a newer kernel version.  Expected output was changed
> to match the new kernel too.

Nack.

Acceptance tests are not to test the latest Linux kernel,
they aim to assert a specific kernel tested by some developer
still works while QEMU evolves.
QEMU doesn't have to adapt to the latest kernel;
QEMU should keep boot an old kernel.

Testing new kernels is good, you are adding coverage. But
this break the acceptance testing contract "keep testing
the same thing over time".

The problem you are trying to fix is the "where to keep
assets from public locations where they are being removed?"
one. Two years ago [*] you suggested to use some storage on
the avocado-project.org:

  For Avocado-VT, there are the JeOS images[1], which we
  keep on a test "assets" directory.  We have a lot of
  storage/bandwidth availability, so it can be used for
  other assets proven to be necessary for tests.

  As long as distribution rights and licensing are not
  issues, we can definitely use the same server for kernels,
  u-boot images and what not.

  [1] - https://avocado-project.org/data/assets/

It is time to have QEMU assets managed the same way.

[*] https://www.mail-archive.com/avocado-devel@redhat.com/msg00860.html

> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 44 +++++++++++++-------------
>  tests/acceptance/replay_kernel.py      |  8 ++---
>  2 files changed, 26 insertions(+), 26 deletions(-)
> 
> diff --git a/tests/acceptance/boot_linux_console.py 
> b/tests/acceptance/boot_linux_console.py
> index aaa781a581..751b47b8fd 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -490,12 +490,12 @@ class BootLinuxConsole(LinuxKernelTest):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:cubieboard
>          """
> -        deb_url = ('https://apt.armbian.com/pool/main/l/'
> -                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> -        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> +        deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> +                   'linux-image-dev-sunxi_20.08_armhf.deb')
> +        deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
>          deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
>          kernel_path = self.extract_from_deb(deb_path,
> -                                            '/boot/vmlinuz-4.20.7-sunxi')
> +                                            '/boot/vmlinuz-5.8.0-sunxi')
>          dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
>          dtb_path = self.extract_from_deb(deb_path, dtb_path)
>          initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> @@ -522,7 +522,7 @@ class BootLinuxConsole(LinuxKernelTest):
>          exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
>                                                  'Allwinner sun4i/sun5i')
>          exec_command_and_wait_for_pattern(self, 'cat /proc/iomem',
> -                                                'system-control@1c00000')
> +                                                '1c00000.system-control')
>          # cubieboard's reboot is not functioning; omit reboot test.
>  
>      def test_arm_cubieboard_sata(self):
> @@ -530,12 +530,12 @@ class BootLinuxConsole(LinuxKernelTest):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:cubieboard
>          """
> -        deb_url = ('https://apt.armbian.com/pool/main/l/'
> -                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> -        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> +        deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> +                   'linux-image-dev-sunxi_20.08_armhf.deb')
> +        deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
>          deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
>          kernel_path = self.extract_from_deb(deb_path,
> -                                            '/boot/vmlinuz-4.20.7-sunxi')
> +                                            '/boot/vmlinuz-5.8.0-sunxi')
>          dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
>          dtb_path = self.extract_from_deb(deb_path, dtb_path)
>          rootfs_url = ('https://github.com/groeck/linux-build-test/raw/'
> @@ -573,12 +573,12 @@ class BootLinuxConsole(LinuxKernelTest):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:orangepi-pc
>          """
> -        deb_url = ('https://apt.armbian.com/pool/main/l/'
> -                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> -        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> +        deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> +                   'linux-image-dev-sunxi_20.08_armhf.deb')
> +        deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
>          deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
>          kernel_path = self.extract_from_deb(deb_path,
> -                                            '/boot/vmlinuz-4.20.7-sunxi')
> +                                            '/boot/vmlinuz-5.8.0-sunxi')
>          dtb_path = '/usr/lib/linux-image-dev-sunxi/sun8i-h3-orangepi-pc.dtb'
>          dtb_path = self.extract_from_deb(deb_path, dtb_path)
>  
> @@ -598,12 +598,12 @@ class BootLinuxConsole(LinuxKernelTest):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:orangepi-pc
>          """
> -        deb_url = ('https://apt.armbian.com/pool/main/l/'
> -                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> -        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> +        deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> +                   'linux-image-dev-sunxi_20.08_armhf.deb')
> +        deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
>          deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
>          kernel_path = self.extract_from_deb(deb_path,
> -                                            '/boot/vmlinuz-4.20.7-sunxi')
> +                                            '/boot/vmlinuz-5.8.0-sunxi')
>          dtb_path = '/usr/lib/linux-image-dev-sunxi/sun8i-h3-orangepi-pc.dtb'
>          dtb_path = self.extract_from_deb(deb_path, dtb_path)
>          initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> @@ -629,7 +629,7 @@ class BootLinuxConsole(LinuxKernelTest):
>          exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
>                                                  'Allwinner sun8i Family')
>          exec_command_and_wait_for_pattern(self, 'cat /proc/iomem',
> -                                                'system-control@1c00000')
> +                                                '1c00000.system-control')
>          exec_command_and_wait_for_pattern(self, 'reboot',
>                                                  'reboot: Restarting system')
>          # Wait for VM to shut down gracefully
> @@ -641,12 +641,12 @@ class BootLinuxConsole(LinuxKernelTest):
>          :avocado: tags=machine:orangepi-pc
>          :avocado: tags=device:sd
>          """
> -        deb_url = ('https://apt.armbian.com/pool/main/l/'
> -                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> -        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> +        deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> +                   'linux-image-dev-sunxi_20.08_armhf.deb')
> +        deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
>          deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
>          kernel_path = self.extract_from_deb(deb_path,
> -                                            '/boot/vmlinuz-4.20.7-sunxi')
> +                                            '/boot/vmlinuz-5.8.0-sunxi')
>          dtb_path = '/usr/lib/linux-image-dev-sunxi/sun8i-h3-orangepi-pc.dtb'
>          dtb_path = self.extract_from_deb(deb_path, dtb_path)
>          rootfs_url = ('http://storage.kernelci.org/images/rootfs/buildroot/'
> diff --git a/tests/acceptance/replay_kernel.py 
> b/tests/acceptance/replay_kernel.py
> index b79fc8daf8..80a460814f 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -132,12 +132,12 @@ class ReplayKernel(LinuxKernelTest):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:cubieboard
>          """
> -        deb_url = ('https://apt.armbian.com/pool/main/l/'
> -                   'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> -        deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> +        deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> +                   'linux-image-dev-sunxi_20.08_armhf.deb')
> +        deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
>          deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
>          kernel_path = self.extract_from_deb(deb_path,
> -                                            '/boot/vmlinuz-4.20.7-sunxi')
> +                                            '/boot/vmlinuz-5.8.0-sunxi')
>          dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
>          dtb_path = self.extract_from_deb(deb_path, dtb_path)
>          initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
>