Re: [Qemu-devel] Question for iotests 188, 189 and 087

[Cleber Rosa]

On 07/18/2017 02:07 PM, John Snow wrote:
> 
> 
> On 07/17/2017 11:01 PM, Jing Liu wrote:
>> Hi all,
>>
>> Do you anybody have iotests failure: 188, 189 and 087 of the latest qemu
>> upsteam?
>>
>> I just wondered if it has something wrong with my test machines because
>> I have different results with two machines.
>>
>>
>> Thanks.
>>
>> Jing
>>
>>
> 
> Presumably you are missing aio=native support for 087,
> 

I see 6 different "tests" as part of 087, and only one them requires AIO
support ("aio=native without O_DIRECT"), which can fail like this:

--- /home/cleber/src/qemu/tests/qemu-iotests/087.out    2017-07-17
19:33:26.409758360 -0400
+++ 087.out.bad 2017-07-18 17:01:37.736038689 -0400
@@ -27,7 +27,7 @@
 Testing:
 QMP_VERSION
 {"return": {}}
-{"error": {"class": "GenericError", "desc": "aio=native was specified,
but it requires cache.direct=on, which was not specified."}}
+{"error": {"class": "GenericError", "desc": "aio=native was specified,
but is not supported in this build."}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "SHUTDOWN", "data": {"guest": false}}

Failures: 087
Failed 1 of 1 tests

When either "--disable-linux-aio" is given or when libaio headers are
missing, which in the end means that CONFIG_LINUX_AIO won't be set in
config-host.mak or config-host.h.

> and 188/189 I believe require compression and/or LUKS libraries.
> 

I did not check if/which optional configure time option would end up
disabling LUKS support and affecting those tests, but it's possible that
something similar exists.

> These are false positives caused by missing libraries. We should
> probably work out a proper solution to meaningfully skipping tests we
> didn't compile support for.
> 
> --js
> 

I see issues here:

1) The qemu-iotests "runner", that is, "./check", understands that a
file number is a test.  No further granularity is (currently?)
achievable here.

The easy solution would be to split tests which depend on optional
features to separate test files.  In the 087 test given as example,
the "=== aio=native without O_DIRECT ===" block would become, say, 190.

2) Skip tests based on features more easily.  There's already support
for skipping tests on various situations.  From 087:

...
_supported_fmt qcow2
_supported_proto file
_supported_os Linux
...

It's trivial to also have access to other compile time settings.  I did
a quick experiment that would add a "_required_feature" function to
"common.rc".  For 087 (or 190?) it would look like:

...
_supported_fmt qcow2
_supported_proto file
_supported_os Linux
_required_feature CONFIG_LINUX_AIO
...

Does it make sense?  Would a patch series along those lines would be
welcomed by reviews?

Thanks!

-- 
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]

signature.asc
Description: OpenPGP digital signature