Re: [PATCH 0/9] Use known good releases when installing in pyvenv

[Philippe Mathieu-Daudé]

Hi Paolo,

On 9/8/23 11:29, Paolo Bonzini wrote:
> This series introduce a new installation command for mkvenv.py that
> retrieves the packages to be installed from a TOML file. This allows
> being more flexible in using the system version of a package, while at
> the same time using a known-good version when installing the package.
> This is important for packages that sometimes have backwards-incompatible
> changes or that depend on specific versions of their dependencies.
>      
> For example, in the case of sphinx we can always use the last version
> that supports older versions of Python and especially docutils (Ubuntu
> 20.04 for example would not support sphinx 6.0), and in the case of
> Avocado we can avoid installing a package that conflicts with any
> plugins already existing in the system package path.  This in turn
> enables using the same virtual environment for both the build and
> qemu-iotests.  John has patches for this.
>
> For the configuration file, TOML was chosen because it was more human
> readable and easier to edit than JSON.  A parser is available in the
> Python 3.11 standard library; for older versions, the API-compatible
> replacement tomli is very small (12k).  I am introducing it as a vendored
> .whl file because it is not installed by default in most distros (unlike
> pip and setuptools which were introduced in 8.0) and because Debian
> 11 only has it in bullseye-backports.  However, if preferred the patch
> "python: use vendored tomli" can be dropped.  In that case, tomli will
> have to be installed from either PyPI or bullseye-backports.
>
> While tomli is bundled with pip, this is only true of recent versions
> of pip.  Of all the supported OSes pretty much only FreeBSD has a recent
> enough version of pip while staying on Python <3.11.  So we cannot use
> the same trick that is in place for distlib.
>
> In order to pick the tomli mapping, lcitool is updated to a recent
> version.  As a side effect this updates from LEAP 15.4 to 15.5.

Does this fix the second point of commit eaf245becd ("Revert
"tests/requirements.txt: bump up avocado-framework version to 101.0"")

 > Second, the recent changes to Python handling mean that there is
 > a single virtual environment for all the build, instead of a separate
 > one for testing.

?

If so, what's left to be able to run QEMU (Avocado) test suite on macOS
is the first point from the same commit:

 > Avocado is not logging of command lines or terminal output, and
 > not collecting Python logs outside the avocado namespace.

This seems to be fixed in Avocado 102.0:

https://avocado-framework.readthedocs.io/en/102.0/releases/102_0.html

 > The root logger handler was restored. This enables all loggers
 > out of avocado.* namespace by default.

Assuming this is enough for macOS, what would be the next change,
moving to 102.0 or integrate 102.0 in the current set? I.e.:

  avocado-framework = { accepted = "(>=88.1, <93.0, 102.0)", ... }

? But I expect the "<93.0" filter to clash...

Thanks,

Phil.