[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Python and propagation
From: |
Federico Beffa |
Subject: |
Re: Python and propagation |
Date: |
Mon, 22 Feb 2016 18:08:16 +0100 |
Ricardo Wurmus <address@hidden> writes:
> 1) print a warning when a collision is expected to happen, not when a
> collision has happened.
+1
> 2) avoid PYTHONPATH, patch all Python files invasively!
>
> Python does not have any feature that is comparable to RUNPATH. It is
> only concerned with finding libraries/modules by *name* in one of the
> directories specified by the PYTHONPATH environment variable.
>
> But actually the PYTHONPATH variable is not the only means to affect the
> search path for modules. It is possible to change the search path
> programmatically:
>
> import sys
> sys.path.append("/gnu/store/cabba9e...-numpy.../lib/...")
> import numpy
>
> The first two lines add an explicit store item path to the search path;
> the third line just imports the numpy module by name (as usual). Even
> without setting the PYTHONPATH to include the numpy in the profile the
> third line won’t fail.
>
> I wonder if we could design a phase that — very much like the
> “wrap-program” phase — modifies *every* Python file and injects lines
> like the first two in the example above, appending explicit store item
> paths, so that all dependent Python libraries can be found without the
> need to have them installed in the same profile and without the need to
> set PYTHONPATH.
>
> Maybe this is crazy, and maybe this causes other annoying problems, but
> I think it is the closest we can get to a RUNPATH-like feature in
> Python.
>
> What do you think? Do I need a long vacation or is this something we
> might realistically do in an automated fashion?
Isn't the problem left of how the python interpreter can find those
patched libraries?
Maybe something along this lines could do:
* Instead of installing all files related to a module into profiles,
only install a .pth file in 'site-packages' with a unique name. The
name could even include the Guix package hash. This file should
include the full path to the store ('real') package. This could maybe
be achieved with 'python-build-system' creating two derivations: a
derivation including only the .pth file and the 'real' package
derivation (similar to a wrapped program pointing to the real one).
For example
$cd ~/tmp/ttmp
$echo -e "aspell\nmechanics" > test.pth
$cd
$python
>>> import sys
>>> import site
>>> site.addsitedir("/home/beffa/tmp/ttmp")
>>> sys.path[-3:]
['/home/beffa/tmp/ttmp', '/home/beffa/tmp/ttmp/aspell',
'/home/beffa/tmp/ttmp/mechanics']
* Then you can install many different versions of a package and they
will not mask each other.
To select a specific version we could then possibly use pkg_resources.
For example:
>>> import pkg_resources
>>> pkg_resources.require("numpy==1.9.1")
[numpy 1.9.1
(/gnu/store/hbdzccnvnlf54mflgcigqw2jv4ybippv-profile/lib/python3.4/site-packages)]
>>> import numpy
If this always works (I'm not sure), we could patch this info into the
packages at compile time.
Regards,
Fede
- Re: Python and propagation, (continued)