Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again

[Vladimir Sementsov-Ogievskiy]

24.04.2019 13:59, Thomas Huth wrote:
> On 24/04/2019 12.48, Daniel P. Berrangé wrote:
>> On Wed, Apr 24, 2019 at 12:37:46PM +0200, Thomas Huth wrote:
>>> People often forget to run the iotests before submitting patches or
>>> pull requests - this is likely due to the fact that we do not run the
>>> tests during our mandatory "make check" tests yet. Now that we've got
>>> a new "ci" group of iotests that should be fine to run in every environ-
>>> ment, it should be OK to enable the iotests during "make check" again.
>>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
>>> script, and only use the former "quick" group (that contains some tests
>>> that are failing in some environments) when the user decided to run
>>> "make check SPEED=thorough" or something similar.
>>>
>>> Signed-off-by: Thomas Huth <address@hidden>
>>> ---
>>>   tests/Makefile.include      |  2 +-
>>>   tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>>>   2 files changed, 17 insertions(+), 2 deletions(-)
>>
>>> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
>>> index 0e554bb972..416b3fc48b 100755
>>> --- a/tests/qemu-iotests-quick.sh
>>> +++ b/tests/qemu-iotests-quick.sh
>>
>> s/-quick//  perhaps, since it no longer only runs the "quick" tests.
>>
>>> @@ -1,8 +1,23 @@
>>>   #!/bin/sh
>>>   
>>> +# Honor the SPEED environment variable, just like we do it for the qtests.
>>> +# The default is to run all tests that still work fine in a CI 
>>> environments,
>>> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
>>> +# tests that are still marked as "quick"
>>> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
>>> +    group=quick
>>
>> When user asks for "slow" tests we run the "quick" tests, which
>> presumably implies...
>>
>>> +else
>>> +    group=ci
>>
>> ..."ci" is quicker than the "quick"  tests !
>>
>> Confused ? We certainly try to ensure that you will be :-)
> 
> ... and there is also tests/check-block.sh (which is not used by "make
> check-block", by the way, but still talks about "make check-block"),
> which is even slower than the "ci" and "quick" group...
> 
> I guess that could be cleaned up, too (why do we need two scripts?) ...
> just not sure how the final result should look like and if it should be
> part of this patch series or rather something separate. Max, Kevin, what
> do you think?
> 
> Maybe we should merge the two scripts, and depending on the SPEED
> variable, we run:
> 
>   quick => "ci" group
>   thorough => "quick" group
>   slow => old behaviour of the check-block.sh script

May be, instead of SPEED, introduce GROUP variable? To make it plain
ci -> ci
quick -> quick
nothing -> default old behavior
?

> 
> ?
> 
>   Thomas
> 


-- 
Best regards,
Vladimir