[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v7 4/8] Acceptance tests: use relative location for tests

From: Philippe Mathieu-Daudé
Subject: Re: [PATCH v7 4/8] Acceptance tests: use relative location for tests
Date: Tue, 12 Nov 2019 19:17:50 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1

On 11/11/19 11:11 PM, Cleber Rosa wrote:
On Mon, Nov 04, 2019 at 07:26:23PM +0100, Philippe Mathieu-Daudé wrote:
On 11/4/19 4:13 PM, Cleber Rosa wrote:
An Avocado Test ID[1] is composed by a number of components, but it
starts with the Test Name, usually a file system location that was
given to the loader.

Because the source directory is being given as a prefix to the
"tests/acceptance" directory containing the acceptance tests, the test
names will needlessly include the directory the user is using to host
the QEMU sources (and/or build tree).

Let's remove the source dir (or a build dir) from the path given to
the test loader.  This should give more constant names, and when using
result servers and databases, it should give the same test names
across executions from different people or from different directories.

Can we strip the full path to directory and only keep the filename in the
database? (Thinking about out-of-tree tests).

Yes, absolutely, but this needs to be done one the Avocado side.  TBH,
I have ideas to make this go even further, such as:

  1) the stripping of the "test_" prefix of the test method

  2) replace the full path to a directory with tests for a "test suite"
     alias (default to the directory name itself)

  3) test suite alias will be persisted on test result such as reports
     or machine, but ommited from the human UI

  4) full path to directory, exact version of test files (git hash) will
     all be metadata and not part of the test ID

Roughly speaking, something which is listed like this currently:

   $ avocado list tests/acceptance/

When executed, would be shown as:

   JOB ID     : fb885e9c3e7dc50534ec380a7e988cbf94233f07
   JOB LOG    : 
    (1/1) acceptance/boot_linux_console.py:BootLinuxConsole.x86_64_pc: PASS 
(2.17 s)

For the particular use case of QEMU, we can also strip the "acceptance/" part (and eventually ".py").

   RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
   JOB TIME   : 2.35 s

How does that sound?

- Cleber.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]