On Fri, 10 Feb 2023 at 16:01, John Snow <jsnow@redhat.com> wrote:
> On Fri, Feb 10, 2023, 5:41 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>> On Fri, 10 Feb 2023 at 00:31, John Snow <jsnow@redhat.com> wrote:
>> This confuses me. We work fine with Python 3.6 today.
>
>
> That won't last - Many tools such as mypy, pylint and flake8 which I use to manage our python codebase have been dropping support for 3.6 and I've had to implement an increasing number of workarounds to help keep it possible to test 3.6 via CI while also ensuring our newest platforms work as dev environments.
Something somewhere seems kind of out-of-sync here. Either:
* Python is deprecating old versions too quickly and
dropping support for them too fast
I think this is true. The Python upstream ecosystem moves extremely fast and breaks backwards compatibility for tooling often. It is sometimes tough to mediate that flow rate difference.
Part of the reason I added that "check-tox" CI test that is currently optional to run is specifically so it can find problems with cutting edge tooling - and it happens often. (It seems like every freeze, I find out a new upstream version of something breaks the testing, like clockwork.)
On the other hand, Python 3.6 did launch some six years ago - 2016-12-22. It's probably reasonable most cutting edge tooling for f37/f38 no longer supports developing for a version this old.
I can't deny that the Python package ecosystem is too cavalier about dropping for support for things too quickly, though.
* CentOS is retaining old versions of Python when it needs to
ship newer ones
I'm not clear on the exact policies for Stream 8/9, but in our traditional releases we'd not ever upgrade this - we'd only do micro updates and backports.
Instead, Stream 8/9 (and OpenSUSE too) offer optional Python versions that install side-by-side.
So sphinx (as packaged as an rpm) continues to use the platform Python - 3.6 - but you *can* launch the "system sphinx" with your newer Python interpreter - if the version of sphinx shipped happens to cope with a newer interpreter.
(If you run `python3.7 -m sphinx`, for example - it might work, but only if your rpm-packaged sphinx isn't using any features of Python that were dropped in 3.7. I'd wager it probably does work most of the time.)
* Our policy for what distros we support is overly lax
and causing us to try to support ancient platforms
that we shouldn't be trying to support
Possibly. Part of the idea behind RHEL8/CentOS8 is that you'll get security fixes, but it should otherwise be a stable platform for production.
Supporting this as a development environment for cutting edge versions does feel like a mismatch in visions at times.
RHEL8 launched 2018-11-14, so is it reasonable to expect that cutting edge versions of QEMU 4 years in the future will build on it no problem?
Meanwhile, OpenSUSE 15.4 released just eight months ago and still ships Python 3.6 as its default/platform Python. They don't ship a distro with a modern Python as default at all... Even the upcoming 15.5 release still appears to use Python 3.6 as the platform default.
(Like RHEL, they offer optional additional interpreters if you want to run upstream software written within the last two years.)
At least QEMU's stated support for RHEL8 runs out in another year or so. I wonder what OpenSUSE is planning to do.
because "use the system version of foo" should not be a big
deal -- it's not like we're trying to support decades-old
hosts here: Centos 8 was released in 2019, which is less than
five years ago.
Yeah, I have nothing cute to say to this. It's unfortunate that it's hard to support RHEL8 and Fedora 37 simultaneously as development environments for our devs.
I do take great pains to reduce disruptions to everyone as I keep the python code afloat. I hope I have been successful these past few years.
However, there are packaged versions of modern Python available for any platform we support that otherwise defaults to 3.6, so it's not a huge affair to continue supporting these older platforms.
It'll just be as if you need one extra dependency on those systems. (Maybe two if you want to build docs.)
I think that trade-off is fair; as it's only a one-time expense to developers still working on older platforms.
> The argument I'm making is:
>
> - CentOS 8 is a supported build platform
> - All platforms *do* have a Python modern enough to allow us to drop 3.6
> - CentOS's repo version of sphinx is hardcoded to use the older 3.6, though
> - You expressed a preference to me in the past to NOT use a pip installed version of sphinx in preference to the system version in "configure"
> - It's still possible to build docs on CentOS 8 after this patchset, you just need a pip version.
> - We've used the justification that our build platform promise does not necessarily extend to docs and tests in the past.
> - So just skip docs building for CentOS 8, only in the CI.
>
> If you believe docs in CI for CentOS 8 is a hard deal breaker, then I want to go back to discussing the possibility of using sphinx versions from pip.
If Python 3.6 is EOL, shouldn't CentOS have shipped an updated
version of Sphinx to go with their updated Python ?
It's EOL upstream, but I suppose the RHEL/CentOS view is that we'll support it with our own security fixes/backports past its upstream death. I don't know the exact versioning policies, but generally they never do a version bump. I don't expect them to ever upgrade the platform default python from 3.6, because it will likely involve hundreds of version bumps to other packages as a result.
I'm not super-happy about dropping the docs build requirement,
but I guess it's probably the least-worst choice.
I think Paolo is advocating for allowing the build to use a pip-installed version of sphinx, and configuring our docker containers to set up that environment.
We'd keep CentOS 8 building docs just fine as a result.
It would mean that individual contributors that wanted to build docs on CentOS 8 would need to install a sphinx version with pip, though.
(The problem with just allowing sphinx to be a black box and continuing to happily use the 3.6-based versions is that we are using QAPIDoc extensions from our own codebase, which would lock those to 3.6. A big motivator for Markus is dropping some 3.6 kludges we're carrying for qapi, so I looked to the opposite solution - nudging Sphinx forward instead.)
Sorry for the essay!
--js