Re: [PATCH 3/3] iotests: distinguish 'skipped' and 'not run' states

[Eric Blake]

On Wed, Sep 06, 2023 at 04:09:17PM +0200, Denis V. Lunev wrote:
> Each particular testcase could skipped intentionally and accidentally.
> For example the test is not designed for a particular image format or
> is not run due to the missed library.
> 
> The latter case is unwanted in reality. Though the discussion has
> revealed that failing the test in such a case would be bad. Thus the
> patch tries to do different thing. It adds additional status for
> the test case - 'skipped' and bound intentinal cases to that state.

intentional

> 
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> CC: Kevin Wolf <kwolf@redhat.com>
> CC: Hanna Reitz <hreitz@redhat.com>
> CC: Eric Blake <eblake@redhat.com>
> CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  tests/qemu-iotests/common.rc     | 23 ++++++++++++++++-------
>  tests/qemu-iotests/iotests.py    | 19 +++++++++++++++----
>  tests/qemu-iotests/testrunner.py | 17 +++++++++++++++--
>  3 files changed, 46 insertions(+), 13 deletions(-)

I agree that this makes it easier to identify skipped tests that can
never be run (not designed for that combination) vs tests that can
plausibly be run by addressing missing dependencies (such as
installing additional libraries, a different file system, ...).

Reviewed-by: Eric Blake <eblake@redhat.com>

Because patch 1 impacts NBD testing, I can queue the series through my
NBD tree if no one else picks it up first.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org