[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 03/15] python: add VERSION file
Re: [PATCH v2 03/15] python: add VERSION file
Mon, 19 Oct 2020 12:13:33 -0400
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0
On 10/19/20 6:02 AM, Daniel P. Berrangé wrote:
On Mon, Oct 19, 2020 at 11:45:09AM +0200, Andrea Bolognani wrote:
On Wed, 2020-10-14 at 10:29 -0400, John Snow wrote:
Python infrastructure as it exists today is not capable reliably of
single-sourcing a package version from a parent directory. The authors
of pip are working to correct this, but as of today this is not possible
to my knowledge.
The problem is that when using pip to build and install a python
package, it copies files over to a temporary directory and performs its
build there. This loses access to any information in the parent
directory, including git itself.
Further, Python versions have a standard (PEP 440) that may or may not
follow QEMU's versioning. In general, it does; but naturally QEMU does
not follow PEP 440. To avoid any automatically-generated conflict, a
manual version file is preferred.
I am proposing:
- Python core tooling synchronizes with the QEMU version directly
(5.2.0, 5.1.1, 5.3.0, etc.)
- In the event that a Python package needs to be updated independently
of the QEMU version, a pre-release alpha version should be preferred,
but *only* after inclusion to the qemu development or stable branches.
e.g. 5.2.0a1, 5.2.0a2, and so on should be preferred prior to 5.2.0's
- The Python core tooling makes absolutely no version compatibility
checks or constraints. It *may* work with releases of QEMU from the
past or future, but it is not required to.
i.e., "qemu.core" will always remain in lock-step with QEMU.
- We reserve the right to split out e.g. qemu.core.qmp to qemu.qmp
and begin indepedently versioning such a package separately from the
QEMU version it accompanies.
I think this need to be considered very carefully.
I'm not overly familiar with the Python ecosystem but it would appear
that, despite PEP 440 not mandating this, many (most?) of the
packages uploaded to PyPi are using semantic versioning.
Semver is the recommended approach, but they explicitly list date
based versioning as a valid alternative
"Semantic versioning is not a suitable choice for all projects,
such as those with a regular time based release cadence and a
deprecation process that provides warnings for a number of
releases prior to removal of a feature."
That paragraph describes QEMU's scenario.
NB, historically we've made arbitrary changes to the python code
since it was not considered public API. If we make it official
public API, then we would actually need to start following our
deprecation process for the python code too.
I think our deprecation process is not tightly compatible with how
Python programmers at-large expect packages to work. Semver is more or
less the norm, despite the fact that it isn't explicitly required.
setting requirements in requirements.txt, setup.[cfg|py], Pipfile, etc
often hinge on a major version, e.g.
qemu >= 5.0
qemu >= 3.0, < 6.0
would both be common forms of describing a requirement.
Pinning specific versions is considered bad form, but in the context of
releasing a package, I often see maintainers hedging their bets and
preventing upgrades across a major version line.
For that reason I am a little weary of adopting the deprecation policy
as it exists in QEMU directly, and would propose a modification for my
- Features must be marked as deprecated
- They must remain in a deprecated state for [at least] 2 releases
- Deprecated features may not be removed until a major version change.
In practice, this modification is a change from "2 releases" to "at
However, I didn't intend to pay any mind to the deprecation policy
"yet", as I have the package metadata listing the package status as
"Alpha", see below:
With that in mind, I think it would be unwise for qemu.* not to do
the same; in particular, using a version number that's not <1.0.0 for
a package that is very much in flux will almost certainly break
people's expectations, and is also not something that you can easily
take back at a later time.
I don't think it is that big a deal, and there is clear benefit to
having the python code version match the QEMU version that it is
the companioon to.
Do you think it's fine if I start versioning at, say, "0.5.2", I could
ignore a deprecation policy for now? I have a lot of changes I want to
make and expect a lot of breaking changes very quickly.
I just wanted to try -- somehow -- to conjure up a relationship to the
QEMU package it's designed to work with.
Ultimately the versioning scheme just impacts on the version string
conditionals people list for their dependancies. Apps consuming QEMU
can handle any of the version schemes without much difference.
Thanks for your input. This is the trickiest part of the process for me.
I believe there is value in distributing these tools for other
developers to help them prototype and experiment with new features, but
realize it's a tightrope walk because we're flying dangerously close to
providing a management utility that needs to care about cross-version
compatibility and so on.
I have no interest in providing stringent cross-version compatibility,
but at the same time, libraries like QMP are not really at risk of
changing all that much, actually. It will *probably* work for most QEMU
I have some further patches where I clean up the scripts in
./scripts/qmp and move them to ./python/qemu/qmp -- and that work is
making me consider a rework to this series. I think that rework matches
the spirit of an earlier suggestion of yours:
- move ./python/qemu/core/qmp.py to ./python/qemu/qmp/qmp.py
- (in a follow-up series) move ./scripts/qmp/* to ./python/qemu/qmp/*
- rename ./python/qemu/core/ to ./python/qemu/machine
(move accel.py, machine.py and qtest.py to a 'machine' pkg.)
In effect, we'd then have:
qemu.machine -- primarily a test interface for QEMU instances
qemu.qmp -- fairly ageless QMP tools/lib for talking to QEMU
and the different levels of support and "use at your own risk" become
slightly more clear between the two packages.
[PATCH v2 04/15] python: add directory structure README.rst files, John Snow, 2020/10/14
[PATCH v2 06/15] python: add pylint exceptions to __init__.py, John Snow, 2020/10/14
[PATCH v2 07/15] python: move pylintrc into setup.cfg, John Snow, 2020/10/14
[PATCH v2 05/15] python: Add pipenv support, John Snow, 2020/10/14
[PATCH v2 08/15] python: add pylint to pipenv, John Snow, 2020/10/14