[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v3 1/5] Add functional/acceptance tests infrastr
From: |
Cleber Rosa |
Subject: |
Re: [Qemu-devel] [PATCH v3 1/5] Add functional/acceptance tests infrastructure |
Date: |
Wed, 30 May 2018 14:15:07 -0400 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 |
On 05/30/2018 08:54 AM, Stefan Hajnoczi wrote:
> On Tue, May 29, 2018 at 03:37:26PM -0400, Cleber Rosa wrote:
>> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
>> index 0ca1a2d4b5..5e090c87fb 100644
>> --- a/docs/devel/testing.rst
>> +++ b/docs/devel/testing.rst
>> @@ -484,3 +484,156 @@ supported. To start the fuzzer, run
>>
>> Alternatively, some command different from "qemu-img info" can be tested, by
>> changing the ``-c`` option.
>> +
>> +Acceptance tests using the Avocado Framework
>> +============================================
>> +
>> +The ``tests/acceptance`` directory hosts functional tests, also known
>> +as acceptance level tests. They're usually higher level, and may
>> +interact with external resources and with various guest operating
>> +systems.
>
> Can you give more concrete examples of what acceptance tests do? This
> will make it easier for readers to understand where Avocado fits in.
>
> Things that come to mind: checking the behavior of QMP commands and
> guest OS commands.
>
Sure. How about:
---
Tests based on ``avocado_qemu.Test`` can easily:
* Customize the command line arguments given to the convenience
``self.vm`` attribute (a QEMUMachine instance)
* Interact with the QEMU monitor, send QMP commands and check
their results
* Interact with the guest OS, using the convenience console device
(which may be useful to assert the effectiveness and correctness of
command line arguments or QMP commands)
* Interact with external data files that accompany the test itself
(see ``avocado.Test.get_data()``)
* Donwload (and cache) remote data files such as firmware and kernel
images
* Have access to a library of guest OS images (by means of the
``avocado.utils.vmimage`` library)
* Make use of various other test related utilities available at the
test class itself and at the utility library:
-
http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test
-
http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
---
>> +To run all tests in the current directory, tagged in a particular way,
>> +run:
>> +
>> +.. code::
>> +
>> + avocado run -t <TAG> .
>
> What is TAG? I guess this identifies a subset of tests to run?
>
I'm trying to avoid too many references to the Avocado documentation, so
this could become:
---
Tests may be classified according to a convention by using docstring
directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests
in the current directory, tagged as "quick", run:
.. code::
avocado run -t quick .
---
>> +QEMUMachine
>> +~~~~~~~~~~~
>> +
>> +The QEMUMachine API should be somewhat familiar to QEMU hackers. It's
>> +used in the Python iotests, device-crash-test and other Python scripts.
>
> This is not useful to someone who isn't a QEMU hacker :).
>
True... so I would ask, why are they wandering around here? :D
Now, seriously, I'll add a bit of extra info here on v4.
> Please explain that QEMUMachine is a class for controlling a QEMU
> instance, including command-line arguments and monitor commands/events.
>
Sure thing.
- Cleber.
- Re: [Qemu-devel] [PATCH v3 3/5] Acceptance tests: add quick VNC tests, (continued)