Re: [Qemu-devel] [PATCH 0/5] Acceptance/functional tests

[Cleber Rosa]

On 05/24/2018 08:58 PM, Cleber Rosa wrote:
> TL;DR
> =====
> 
> Another version, with a minimalist approach, to the acceptance tests
> infrastructure for QEMU, based on the Avocado Testing Framework.
> 
> Background
> ==========
> 
> The previous version, still considered an RFC, was sent to the list by
> Eduardo[1] was based on the work held in Amador's branch[2].  After
> reviewing in under a different light, including the experiences
> done and reported by Philippe[3].
> 

(major sigh for killing a line, writing a non-sense sentence)

... it was clear to me that a different approach would be better.

- Cleber.

> Differences from previous versions
> ==================================
> 
> The main difference is that this series include only the minimal
> changes deemed necessary to have a starting point.  I like to think
> that it's better connected to the QEMU community and project needs,
> and will hopefully allow for a more organic growth.
> 
> Since this version has less features than the previous versions,
> provided it's accepted, these are the next probable development tasks:
> 
>  * Provide a simple variants mechanism to allow the same tests to be
>    run under different targets, machine models and devices (present on
>    the previous versions as a "YAML to Mux" file with architecture
>    definitions)
>  * Implement QEMUMachine migration support (present on the previous
>    version in the "avocado_qemu.test._VM" class)
>  * Implement Guest OS image selection and download (mostly an Avocado
>    feature, paired with a parameter convention and cloud-init support
>    code)
>  * Implement interactive support for Guest OS sessions (present on
>    the previous versions, supported by the aexpect Python module)
> 
> Even though this version shares very little (if any) code with the
> previous versions, the following is a list of noteworthy changes:
> 
>  * Tests directory is now "tests/acceptance" (was "tests/avocado")
>  * Base test class is now "avocado_qemu.Test" (was
>    "avocado_qemu.test.QemuTest")
>  * Base test class is now hosted in "avocado_qemu/__init__.py" (was
>    "avocado_qemu/test.py")
>  * Direct use of "qemu.QEMUMachine", that is, the
>    avocado_qemu.test._VM class is gone
>  * avocado_qemu.Test won't search for QEMU binaries on $PATH.  To use
>    QEMU binary on a custom system location it's necessary to use the
>    "qemu_bin" parameter
>  * Example test in README.rst is distributed as a real test
>    ("test_version.py")
>  * A new "Linux boot console" test, loosely modeled after Phillipe's
>    use case
> 
> Commit summary
> ==============
> 
> Cleber Rosa (5):
>   Add functional/acceptance tests infrastructure
>   scripts/qemu.py: allow adding to the list of extra arguments
>   Acceptance tests: add quick VNC tests
>   scripts/qemu.py: introduce set_console() method
>   Acceptance tests: add Linux kernel boot and console checking test
> 
>  scripts/qemu.py                             | 103 +++++++++++++++-
>  scripts/test_qemu.py                        | 176 +++++++++++++++++++++++++++
>  tests/acceptance/README.rst                 | 141 +++++++++++++++++++++
>  tests/acceptance/avocado_qemu/__init__.py   |  45 +++++++
>  tests/acceptance/test_boot_linux_console.py |  37 ++++++
>  tests/acceptance/test_version.py            |  13 ++
>  tests/acceptance/test_vnc.py                |  50 ++++++++
>  7 files changed, 564 insertions(+), 1 deletion(-)
>  create mode 100644 scripts/test_qemu.py
>  create mode 100644 tests/acceptance/README.rst
>  create mode 100644 tests/acceptance/avocado_qemu/__init__.py
>  create mode 100644 tests/acceptance/test_boot_linux_console.py
>  create mode 100644 tests/acceptance/test_version.py
>  create mode 100644 tests/acceptance/test_vnc.py
> 
> ---
> 
> [1] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03443.html
> [2] https://github.com/apahim/qemu/commits/avocado_qemu
> [3] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03076.html
> 
>