[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v4 0/3] Bootstrap Python venv and acceptance/fun
From: |
Philippe Mathieu-Daudé |
Subject: |
Re: [Qemu-devel] [PATCH v4 0/3] Bootstrap Python venv and acceptance/functional tests |
Date: |
Wed, 17 Oct 2018 12:20:04 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0 |
On 16/10/2018 16:27, Cleber Rosa wrote:
> On 10/12/18 5:44 PM, Philippe Mathieu-Daudé wrote:
>> On 12/10/2018 18:53, Cleber Rosa wrote:
>>> TL;DR
>>> =====
>>>
>>> Allow acceptance tests to be run with `make check-acceptance`.
>>>
>>> Details
>>> =======
>>>
>>> This introduces a Python virtual environment that will be setup within
>>> the QEMU build directory, that will contain the exact environment that
>>> tests may require.
>>>
>>> There's one current caveat: it requires Python 3, as it's based on the
>>> venv module. This was based on some discussions and perception about
>>> standardizing on Python 3, but can easily be made to accommodate Python
>>> 2 as well.
>>>
>>> Example of bootstrap and test execution on Travis-CI:
>>>
>>> https://travis-ci.org/qemu/qemu/jobs/439331028#L2508
>>
>> If you activate Travis on your github account, you can test that in your
>> namespace without having to open zombie pull requests there... A simple
>> push to your repository will trigger a full Travis build.
>>
>> This is how I use it btw, canceling the jobs I'm not interested in, to
>> quickly run the others.
>> i.e. https://travis-ci.org/philmd/qemu/jobs/439573299#L5600
>>
>
> Yep, I've done that shortly after sending this. :) Thanks for the tip,
> though.
>
> One thing I miss is the ability, without editing the .travis.yaml, to
> "choose" just one/few jobs in a build. Or maybe even the order they run
> (Of course I want the acceptance tests job first).
>
> Do you know anything about that?
I just add an extra commit that drop every tests but the ones I'm
interested to run and keep this patch until I'm happy with the series.
I push as 'myseries_only' then I simply do "git branch myseries_only~
myseries && git push series" to run against everything, go to check my
mailbox and review patches until it finishes, then I redact the cover
and post.
>
> - Cleber.
>
>>>
>>> ...
>>> VENV /home/travis/build/qemu/qemu/tests/venv
>>> MKDIR /home/travis/build/qemu/qemu/tests/results
>>> PIP /home/travis/build/qemu/qemu/tests/venv-requirements.txt
>>> AVOCADO tests/acceptance
>>> JOB ID : 920e4fcf55a1782f1ae77bee64b20ccdc2e1111d
>>> JOB LOG :
>>> /home/travis/build/qemu/qemu/tests/results/job-2018-10-09T21.42-920e4fc/job.log
>>> (1/6)
>>> /home/travis/build/qemu/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test:
>>> PASS (3.57 s)
>>> (2/6)
>>> /home/travis/build/qemu/qemu/tests/acceptance/version.py:Version.test_qmp_human_info_version:
>>> PASS (0.04 s)
>>> (3/6)
>>> /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS
>>> (0.04 s)
>>> (4/6)
>>> /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:
>>> PASS (0.04 s)
>>> (5/6)
>>> /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:
>>> PASS (0.04 s)
>>> (6/6)
>>> /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_vnc_change_password:
>>> PASS (0.04 s)
>>> RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
>>> | CANCEL 0
>>> JOB TIME : 3.90 s
>>> ...
>>>
>>> Changes from v3:
>>> ================
>>>
>>> * Fixed typo in commit message (s/requiment/requirement/). (Eric)
>>>
>>> Changes from v2:
>>> ================
>>>
>>> * Make the $(TESTS_VENV_DIR) target depend on the
>>> venv-requirements.txt file, and touch $(TESTS_VENV_DIR) after venv
>>> runs. With this, updates on the file are reflected on the
>>> venv. (Philippe)
>>>
>>> * Run pip with "python -m pip". It may have been installed reusing
>>> the system wide packages, and then the script may not be available
>>> on the venv. (Philippe)
>>>
>>> * Dropped Python version on Travis, and using the version supplied
>>> by the distro (3.4). (Philippe)
>>>
>>> * Added "python3.4-venv" package requirement on Travis. (Philippe)
>>>
>>> * Added variable (AVOCADO_SHOW) with logging streams to be shown
>>> while running the acceptance tests. By default it's set to none,
>>> the equivalent of the quiet mode used on previous versions.
>>> (Philippe)
>>>
>>> * On Travis, set the AVOCADO_SHOW variable to "app", so that the
>>> individual test results can be easily seen. (Philippe)
>>>
>>> Ideas discussed, but not implemented:
>>>
>>> * Run pip with "$(PYTHON) -m pip -q install ..." because it points
>>> to the system wide Python installation. (Philippe)
>>>
>>> * Drop the "--system-site-packages" flag. Waiting on another round
>>> of tests to determine if they are really the cause of some package
>>> installation problems.
>>>
>>> Changes from v1:
>>> ================
>>>
>>> * TESTS_VENV_REQ (the path of "venv-requirements.txt") now points to
>>> the source path ($SRC_PATH instead of $BUILD_DIR)
>>>
>>> * Create the venv with "--system-site-packages", which allows the
>>> reuse of packages (and no additional downloads) in case there's a
>>> package installed system wide providing the same package and
>>> version.
>>>
>>> * Run Avocado with "python -m avocado". It may have been installed
>>> reusing the system wide packages, and then the script may not
>>> be available on the venv.
>>>
>>> * Improved documentation describing the Python 3, venv and pip
>>> requirements.
>>>
>>> * Updated avocado-framework requirement to latest released version
>>> (65.0)
>>>
>>> * (New commit) Added support for running the acceptance tests on
>>> Travis.
>>>
>>> Ideas discussed, but not implemented:
>>>
>>> * Install external packages such as python3-pip on Debian based
>>> systems, deemed too invasive on developer's systems.
>>>
>>> * Allow the use of Python 2, and consequently the "virtualenv"
>>> module.
>>>
>>> Cleber Rosa (3):
>>> Bootstrap Python venv for tests
>>> Acceptance tests: add make rule for running them
>>> Travis support for the acceptance tests
>>>
>>> .travis.yml | 5 +++++
>>> docs/devel/testing.rst | 35 ++++++++++++++++++++++++++++++-----
>>> tests/Makefile.include | 37 +++++++++++++++++++++++++++++++++++++
>>> tests/venv-requirements.txt | 4 ++++
>>> 4 files changed, 76 insertions(+), 5 deletions(-)
>>> create mode 100644 tests/venv-requirements.txt
>>>
>>
>
[Qemu-devel] [PATCH v4 3/3] Travis support for the acceptance tests, Cleber Rosa, 2018/10/12
[Qemu-devel] [PATCH v4 2/3] Acceptance tests: add make rule for running them, Cleber Rosa, 2018/10/12
Re: [Qemu-devel] [PATCH v4 0/3] Bootstrap Python venv and acceptance/functional tests, Philippe Mathieu-Daudé, 2018/10/12