Re: [PATCH v2 0/7] Python: Drop support for Python 3.6

[Daniel P . Berrangé]

On Fri, Feb 17, 2023 at 10:06:49AM +0100, Markus Armbruster wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
> > On 15/02/2023 20.05, Markus Armbruster wrote:
> >> The discussion under PATCH 6 makes me think there's a bit of confusion
> >> about the actual impact of dropping support for Python 3.6.  Possibly
> >> because it's spelled out in the commit message of PATCH 7.  Let me
> >> summarize it in one sentence:
> >> 
> >>      *** All supported host systems continue to work ***
> >> 
> >> Evidence: CI remains green.
> >
> > The CI remains green since one of the patches disabled the building of the 
> > docs on CentOS 8. That's not how I'd describe "continue to work", at least 
> > not in the same extend as before.
> 
> Thus the exception ...
> 
> >> On some supported host systems, different packages need to be installed.
> >> On CentOS 8, for instance, we need to install Python 3.8.13 or 3.9.16
> >> instead of 3.6.8.  Let me stress again: same repository, different
> >> package.  No downsides I can see.
> 
> ... right here:
> 
> >> The *one* exception is Sphinx on CentOS 8.  CentOS 8 does not ship a
> >> version of Sphinx that works with Python 3.7 or newer.  This series
> >> proposes to simply stop building the docs there, unless the user
> >> provides a suitable version of Sphinx (which is easy enough with pip).
> >
> > I think we've all understood that. The thing that you obviously did not 
> > understood: This breaks our support statement.
> > I'm pretty sure that you could also build the whole QEMU suite successfully 
> > on an ancient CentOS 7 or Ubuntu 18.04 system if you manually install a 
> > newer version of GCC and some of the required libraries first. But that's 
> > not how we understand our support statement.
> >
> > Sure, you can argue that you can use "pip install" to get a newer version 
> > of 
> > Sphinx on RHEL 8 / CentOS 8 to continue building the docs there - but is 
> > that really that much different from installing a newer version of GCC and 
> > libraries on an ancient distro that we do not officially support anymore? 
> > I'd say no. You also have to consider that not every build host has access 
> > to the internet, maybe some companies only have an internal mirror of the 
> > distro packages in their intranet (I remember some discussion about such a 
> > case in the past) - so while you were perfectly fine to build the whole of 
> > QEMU on a CentOS 8 there before this change, you could now not build parts 
> > of QEMU anymore there due to the missing possibility to run "pip install" 
> > without full internet connection.
> >
> > And sure, you can argue that it's "just" the documentation. But IMHO that's 
> > still an essential part of the QEMU build, and it used to work before, so 
> > it 
> > feels wrong to cut that now out. And also, if we start with the 
> > documentation now, what's next? If for example scripts/shaderinclude.py 
> > stops working with Python 3.6, do we then simply say: "Oh, it's fine, you 
> > can still build all the other targets that work without this script, just 
> > not the ones anymore that need it"?
> 
> My view on all this is a bit more pragmatic.
> 
> For a human developer, the difference between "dnf install
> python-sphinx" and "pip install sphinx" is, in my opinion, close to
> negligible.  Really no comparison to "git-clone GCC and bootstap it".
> You seem to disagree with that.

The command for installing the package is certainly not
significantly different.

What I think is lacking with pip is what happens after that
point - the ongoing management of the installed packages, or
rather than lack thereof.

If doing user installs, my $HOME/.local/lib ends up with multiple
orphaned previously installed versions. After each python version
upgrade, pip ceases to report about existance previously installed
bits at all, alone ever upgrade them.

Then there is the issue that pip will refuse to install a package,
if it already exists either from pip or the distro install, even
if the version that already exists doesn't satisfy the version
required. This would impact if the user has 'flake8'  N installed,
and QEMU wanted version N+1 installed, both using the same python
runtime version.

This is not the situation we're in with centos8/docutils, but
it is plausible in future once we decide we'll no longer rely
on exclusively on distro provided python packages, and instead
use pip if we ever want a newer version of something.


The python answer to this IIUC, is to use virtualenvs to isolate
everything. If we use keep the virtualenv under the QEMU build
root, then any time you clean your build root the virtualenv
content needs re-populating. At least pip has the sense to use
a cache, so $HOME/.cache/pip keeps a copy of everything you've
previously downloaded (presumably they age out the cache at
some time/size?), to avoid spending time re-loading all the
time. 


NB, I'm not saying the above is a blocker against use of pip.
Just that there are real differences between use of pip and
the distro packaging tools, beyond the initial install command.


If we're going to use pip, then using virtualenvs looks likely
to be very desirable to avoid conflicts between versions of
pkgs QEMU wants vs what the user might already have installed,
as well as to avoid leaving cruft behind in $HOME/.local/lib.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|