Re: [PATCH v3 00/10] Introduce new acpi/smbios avocado tests using biosbits

[Ani Sinha]

On Mon, Oct 10, 2022 at 1:24 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> Please see the README file added in patch 10 for more details.
> Sample runs are as follows:
>
> $ ./tests/venv/bin/avocado run -t acpi tests/avocado --tap -
> ERROR:  Missing parentheses in call to 'print'. Did you mean print(...)? 
> (smbios.py, line 92)
> ERROR:  Missing parentheses in call to 'print'. Did you mean print(...)? 
> (smilatency.py, line 47)
> ERROR:  Missing parentheses in call to 'print'. Did you mean print(...)? 
> (testacpi.py, line 158)

None of the above files are avocado tests or avocado related python
scripts. They are run from within bits in a python 2.7 environment. I
could not find a mechanism to exclude a directory from avocado tests.
I also do not think making those scripts python 3 compliant is a good
use of my time since upgrading bits to use python 3 would be a major
task unrelated to QEMU testing.

> 1..1
> ok 1 tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
>
> $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> ERROR:  Missing parentheses in call to 'print'. Did you mean print(...)? 
> (smbios.py, line 92)
> ERROR:  Missing parentheses in call to 'print'. Did you mean print(...)? 
> (smilatency.py, line 47)
> ERROR:  Missing parentheses in call to 'print'. Did you mean print(...)? 
> (testacpi.py, line 158)
> Fetching asset from 
> tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID     : c06a9feb423bcda5de89bb51353c6c1718719f14
> JOB LOG    : 
> /home/anisinha/avocado/job-results/job-2022-10-10T13.12-c06a9fe/job.log
>  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS 
> (34.11 s)
> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
> CANCEL 0
> JOB TIME   : 37.40 s
>
>  ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py
> Fetching asset from 
> tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID     : a2476dd3fafe289c5e6475f447bc8369f77d57ba
> JOB LOG    : 
> /home/anisinha/avocado/job-results/job-2022-10-10T13.14-a2476dd/job.log
>  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS 
> (34.07 s)
> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
> CANCEL 0
> JOB TIME   : 40.20 s
>
>
> Changelog:
> v3:
>   - converted the existing test to avocado tests as per the popular
>     recommendation. Added appropriate tags.
>   - artifact download URL modified based on gitlab-CI URL.
>
>   For biosbits repo:
>   - moved to a gitlab repo with me being the maintainer.
>   - added .gitlab-ci.yml file to generate the artifacts.
> v2:
>  - a new class of python based tests introduced that is separate from avocado
>    tests or qtests. Can be run by using "make check-pytest".
>  - acpi biosbits tests are the first tests to use pytest environment.
>  - bios bits tests now download the bits binary archives from a remote
>    repository if they are not found locally. The test skips if download
>    fails.
>  - A new environment variable is introduced that can be passed by the tester
>    to specify the location of the bits archives locally. test skips if the
>    bits binaries are not found in that location.
>  - if pip install of python module fails for whatever reaoson, the test skips.
>  - misc code fixes including spell check of the README doc. README has been
>    updated as well.
>  - addition of SPDX license headers to bits test files.
>  - update MAINTAINERS to reflect the new pytest test class.
>
> For biosbits repo:
>  - added Dockerfile and build script. Made bios bits build on gcc 11.
>    https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile
>    https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
>    The build script generates the zip archive and tarball used by the test.
>
> v1: initial patchset. uses qtest to implement the bios bits tests.
>
> Cc: Qemu Devel <qemu-devel@nongnu.org>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Maydell Peter <peter.maydell@linaro.org>
> Cc: John Snow <jsnow@redhat.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Cc: Igor M <imammedo@redhat.com>
> Cc: Michael Tsirkin <mst@redhat.com>
>
> Ani Sinha (10):
>   acpi/tests/avocado/bits: initial commit of test scripts that are run
>     by biosbits
>   acpi/tests/avocado/bits: add SPDX license identifiers for bios bits
>     tests
>   acpi/tests/avocado/bits: disable acpi PSS tests that are failing in
>     biosbits
>   acpi/tests/avocado/bits: add smilatency test suite from bits in order
>     to disable it
>   acpi/tests/avocado/bits: add SPDX license identifiers for bios bits
>     smilatency tests
>   acpi/tests/avocado/bits: disable smilatency test since it does not
>     pass everytime
>   acpi/tests/avocado/bits: add biosbits config file for running bios
>     tests
>   acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses
>     biosbits
>   MAINTAINERS: add myself as the maintainer for acpi biosbits avocado
>     tests
>   acpi/tests/avocado/bits: add a README file
>
>  MAINTAINERS                                   |    6 +
>  tests/avocado/acpi-bits.py                    |  334 +++
>  tests/avocado/acpi-bits/README                |   90 +
>  .../acpi-bits/bits-config/bits-cfg.txt        |   18 +
>  tests/avocado/acpi-bits/bits-tests/smbios.py  | 2434 +++++++++++++++++
>  .../acpi-bits/bits-tests/smilatency.py        |  107 +
>  .../avocado/acpi-bits/bits-tests/testacpi.py  |  287 ++
>  .../avocado/acpi-bits/bits-tests/testcpuid.py |   87 +
>  8 files changed, 3363 insertions(+)
>  create mode 100644 tests/avocado/acpi-bits.py
>  create mode 100644 tests/avocado/acpi-bits/README
>  create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/smilatency.py
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py
>
> --
> 2.34.1
>