Re: [PATCH v7 4/8] Acceptance tests: use relative location for tests

[Cleber Rosa]

On Mon, Nov 04, 2019 at 07:26:23PM +0100, Philippe Mathieu-Daudé wrote:
> On 11/4/19 4:13 PM, Cleber Rosa wrote:
> > An Avocado Test ID[1] is composed by a number of components, but it
> > starts with the Test Name, usually a file system location that was
> > given to the loader.
> > 
> > Because the source directory is being given as a prefix to the
> > "tests/acceptance" directory containing the acceptance tests, the test
> > names will needlessly include the directory the user is using to host
> > the QEMU sources (and/or build tree).
> > 
> > Let's remove the source dir (or a build dir) from the path given to
> > the test loader.  This should give more constant names, and when using
> > result servers and databases, it should give the same test names
> > across executions from different people or from different directories.
> 
> Can we strip the full path to directory and only keep the filename in the
> database? (Thinking about out-of-tree tests).
>

Yes, absolutely, but this needs to be done one the Avocado side.  TBH,
I have ideas to make this go even further, such as:

 1) the stripping of the "test_" prefix of the test method

 2) replace the full path to a directory with tests for a "test suite"
    alias (default to the directory name itself)

 3) test suite alias will be persisted on test result such as reports
    or machine, but ommited from the human UI

 4) full path to directory, exact version of test files (git hash) will
    all be metadata and not part of the test ID

Roughly speaking, something which is listed like this currently:

  $ avocado list tests/acceptance/
  INSTRUMENTED 
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc
  ...

When executed, would be shown as:

  JOB ID     : fb885e9c3e7dc50534ec380a7e988cbf94233f07
  JOB LOG    : 
/home/cleber/avocado/job-results/job-2019-11-11T17.07-fb885e9/job.log
   (1/1) acceptance/boot_linux_console.py:BootLinuxConsole.x86_64_pc: PASS 
(2.17 s)
  RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
CANCEL 0
  JOB TIME   : 2.35 s

How does that sound?

- Cleber.