Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job

[Philippe Mathieu-Daudé]

On 06/28/2018 02:13 PM, Alex Bennée wrote:
> 
> Cleber Rosa <address@hidden> writes:
> 
>> On 06/28/2018 11:46 AM, Alex Bennée wrote:
>>>
>>> Philippe Mathieu-Daudé <address@hidden> writes:
>>>
>>>> Hi,
>>>>
>>>> Another neanderthal approach to add multi-arch acceptance tests using
>>>> Avocado.
>>>>
>>>> Since Cleber Rosa work got merged [0], I can restart my previous attempt 
>>>> [1]
>>>> at using Avocado in QEMU.
>>>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
>>>> a bit further:
>>>> - we can now run other archs
>>>> - Avocado is integrated into Travis CI
>>>>
>>>> I'm not sure Travis is the best CI available for those tests, but it runs
>>>> quite fast:
>>>>
>>>>       ...
>>>>       LINK    x86_64-softmmu/qemu-system-x86_64
>>>>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>>>>     JOB LOG    : 
>>>> /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>>>>      (1/7) 
>>>> tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS 
>>>> (3.89 s)
>>>>      (2/7) 
>>>> tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS 
>>>> (1.83 s)
>>>>      (3/7) 
>>>> tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS 
>>>> (0.05 s)
>>>>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>>>>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS 
>>>> (0.04 s)
>>>>      (6/7) 
>>>> tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  
>>>> PASS (0.05 s)
>>>>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS 
>>>> (0.05 s)
>>>>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 
>>>> | CANCEL 0
>>>>     JOB TIME   : 6.21 s
>>>
>>> Hmm however if I do:
>>>
>>>    pip install --user avocado-framework
>>>    avocado run tests/acceptance
>>>
>>> I get:
>>>
>>>    Failed to load plugin from module "avocado_vt.plugins.vt_list": 
>>> ImportError('No module named netaddr',)
>>>    Failed to load plugin from module "avocado_vt.plugins.vt": 
>>> ImportError('No module named netaddr',)
>>>    Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": 
>>> OSError(13, 'Permission denied')
>>
>> Alex,
>>
>> These are errors related to your installation of Avocado-*VT*[1].
>>
>>>    Error running method "run" of plugin "virt": 'Namespace' object has no 
>>> attribute 'default_avocado_params'
>>>    Avocado crashed unexpectedly: [Errno 13] Permission denied: 
>>> '/var/lib/libvirt/images/avocado_sg6NRi'
>>>    You can find details in 
>>> /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log
>>>
>>
>> And these from Avocado-*Virt*[2].  Neither of those are used in the
>> tests under "tests/acceptance".
>>
>>> Which I seem to remember from last time I looked into this. I think
>>> avocado makes a bunch of assumptions about the filesystem and OS layout
>>> which is hard to gloss over for something you should be able to run
>>> straight out of the source tree.
>>>
>>>
>>
>> These Avocado-*VT* issues deserve a better look indeed.  Would you
>> please describe (if not here, under a GH issue) your environment?
> 
> Ahh probably from last time I tried to get things up and running. I
> rm -rf ~/.local and things seem to be working well now.

Should we check for a prerequisite version?

> 
> However I was under the impression that VT or virt was needed for this
> testing. Or is this an old assumption?

I don't use neither of these for my qemu-system testing, but I'm more
interested in testing system targets that are different from my host.

I think the VT/virt make sens when you test on a same (target, host)
tuple and worry about perfs.

<snip>