Re: [PATCH v3 02/15] python: add qemu package installer

[John Snow]

On 10/28/20 11:10 AM, Cleber Rosa wrote:
> On Tue, Oct 20, 2020 at 03:35:42PM -0400, John Snow wrote:
> > Add setup.cfg and setup.py, necessary for installing a package via
> > pip. Add a rst document explaining the basics of what this package is
> > for and who to contact for more information. This document will be used
> > as the landing page for the package on PyPI.
> >
> > I am not yet using a pyproject.toml style package manifest, because
> > using pyproject.toml (and PEP-517) style packages means that pip is not
> > able to install in "editable" or "develop" mode, which I consider
> > necessary for the development of this package.
> >
> > Use a light-weight setup.py instead.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> >   python/PACKAGE.rst | 26 ++++++++++++++++++++++++++
> >   python/setup.cfg   | 18 ++++++++++++++++++
> >   python/setup.py    | 23 +++++++++++++++++++++++
> >   3 files changed, 67 insertions(+)
> >   create mode 100644 python/PACKAGE.rst
> >   create mode 100755 python/setup.cfg
> >   create mode 100755 python/setup.py
> >
> > diff --git a/python/PACKAGE.rst b/python/PACKAGE.rst
> > new file mode 100644
> > index 0000000000..b82f32f489
> > --- /dev/null
> > +++ b/python/PACKAGE.rst
> > @@ -0,0 +1,26 @@
> > +QEMU Python Tooling
> > +===================
> > +
> > +This package provides QEMU tooling used by the QEMU project to build,
> > +configure, and test QEMU. It is not a fully-fledged SDK and it is subject
> > +to change at any time.
> > +
> > +Usage
> > +-----
> > +
> > +The ``qemu.qmp`` subpackage provides a library for communicating with
> > +QMP servers. The ``qemu.machine`` subpackage offers rudimentary
> > +facilities for launching and managing QEMU processes. Refer to each
> > +pacakge's documentation
>
> Typo here: s/pacakge/package/

oops,

> > +(``>>> help(qemu.qmp)``, ``>>> help(qemu.machine)``)
> > +for more information.
> > +
> > +Contributing
> > +------------
> > +
> > +This package is maintained by John Snow <jsnow@redhat.com> as part of
> > +the QEMU source tree. Contributions are welcome and follow the `QEMU
> > +patch submission process
> > +<https://wiki.qemu.org/Contribute/SubmitAPatch>`_. There is a `Gitlab
>
> Git*L*ab.  Given that we're also under a company named with two words,
> better given them that :)

They can send me a paycheck if they want me to adhere to their brand 
standards!

(But, OK.)

> > +mirror <https://gitlab.com/jsnow/qemu/-/tree/python>`_, but
> > +contributions must be sent to the list for inclusion.
>
> IMO it's not clear if this branch/mirror is your development work, a
> staging area, etc.

Fair enough. jsnow/qemu/python is intended as a staging area for patches 
that have been vetted on-list.

jsnow/qemu/master is a lazily-updated mirror. (I tend to update it every 
day as part of my development process, but there are days I don't write 
code.)

jsnow/qemu/python-* is development work; review branches, etc.

I'll try to rephrase this a bit. What I want to communicate:

- This package exists as a subfolder of a larger project
- I am responsible for maintaining this package, but not for the larger 
project
- Please contact *me* for problems with this package
- Contributions should go through qemu-devel (I will gently redirect 
contributors who may send pull requests to the qemu devel list.)

> > diff --git a/python/setup.cfg b/python/setup.cfg
> > new file mode 100755
> > index 0000000000..12b99a796e
> > --- /dev/null
> > +++ b/python/setup.cfg
> > @@ -0,0 +1,18 @@
> > +[metadata]
> > +name = qemu
> > +maintainer = QEMU Developer Team
> > +maintainer_email = qemu-devel@nongnu.org
> > +url = https://www.qemu.org/
> > +download_url = https://www.qemu.org/download/
> > +description = QEMU Python Build, Debug and SDK tooling.
> > +long_description = file:PACKAGE.rst
> > +long_description_content_type = text/x-rst
> > +classifiers =
> > +    Development Status :: 3 - Alpha
> > +    License :: OSI Approved :: GNU General Public License v2 (GPLv2)
> > +    Natural Language :: English
> > +    Operating System :: OS Independent
> > +

Also ... licensing question, do we need *L*GPLv2, or does Python not 
have a "linking exception" problem?

I guess we can't really re-license these files anyway, so nevermind.

(I immediately regret asking this.)

> I know the sky is the limit, but I miss the Python version classifier,
> at least:
>
>    Programming Language :: Python :: 3 :: Only

Sure.

Wait, why can you specify Python as a language? Is it possible to have 
non-Python packages on PyPI?

*CONCERNED*

> And optionally those:
>
>    Programming Language :: Python :: 3.6
>    Programming Language :: Python :: 3.7
>    Programming Language :: Python :: 3.8
>    Programming Language :: Python :: 3.9
>
> Although it may be a good idea to add them along test jobs on those
> specific Python versions.

Are these worth adding? I've got python_requires >= 3.6 down below. From 
my test of a blank package upload to PyPI, it seems to display prominently:

https://pypi.org/project/qemu/

Is there a tangible benefit that you are aware of?

> > +[options]
> > +python_requires = >= 3.6
> > +packages = find_namespace:
> > diff --git a/python/setup.py b/python/setup.py
> > new file mode 100755
> > index 0000000000..e93d0075d1
> > --- /dev/null
> > +++ b/python/setup.py
> > @@ -0,0 +1,23 @@
> > +#!/usr/bin/env python3
> > +"""
> > +QEMU tooling installer script
> > +Copyright (c) 2020 John Snow for Red Hat, Inc.
> > +"""
> > +
> > +import setuptools
> > +import pkg_resources
> > +
> > +
> > +def main():
> > +    """
> > +    QEMU tooling installer
> > +    """
> > +
> > +    # 
> > https://medium.com/@daveshawley/safely-using-setup-cfg-for-metadata-1babbe54c108
> > +    pkg_resources.require('setuptools>=39.2')
>
> Getting back to the "test jobs on those specific Python versions" I
> was really anxious that environments with Python 3.6 will fail to
> have such a "recent" setuptools version.

Reasonable doubt. However, this isn't *required* to use the library (the 
QEMU code can continue to just set PYTHONPATH or sys.path as necessary) 
and bypasses the installer entirely.

That gives us some leeway apart from the usual version constraints; in 
order to independently use this library outside of the QEMU tree we may 
impose more modern setups -- as long as the minimum requirements for 
QEMU itself don't break.

Having a modern setuptools in order to install seems like less of a 
problem barrier; and it seemed like a good idea to make it explicitly 
fail instead of silently doing something weird if it didn't 
see/understand setup.cfg.

(And it seems like good practice to update pip in your venv, so I think 
we'll be OK except for the stodgiest of users, but sometimes you can't 
have new things on old systems without learning some new tricks!)

> CentOS 8 has that specific version, while Ubuntu 18.04 has version
> 39.0.  Ubuntu 20.04 has a recent enough version though.  Given that
> all GitLab CI moved to 20.04, this should be safe.
>
> - Cleber.

FWIW, for the purposes of running the linters, I am using Fedora32 and 
the python36 package.

> > +
> > +    setuptools.setup()
> > +
> > +
> > +if __name__ == '__main__':
> > +    main()
> > --
> > 2.26.2