[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
08/104: guix: python-build-system: Add background about Python installat
08/104: guix: python-build-system: Add background about Python installation methods.
Tue, 15 Nov 2016 21:36:59 +0000 (UTC)
htgoebel pushed a commit to branch python-build-system
in repository guix.
Author: Hartmut Goebel <address@hidden>
Date: Tue Nov 15 16:57:21 2016 +0100
guix: python-build-system: Add background about Python installation methods.
guix/build/python-build-system.scm | 68 +++++++++++++++++++++++++++++++++++-
1 file changed, 67 insertions(+), 1 deletion(-)
diff --git a/guix/build/python-build-system.scm
index 310ba8a..3f280b0 100644
@@ -36,7 +36,70 @@
;; Builder-side code of the standard Python package build procedure.
+;; Backgound about the Python installation methods
+;; In Python there are different ways to install packages: distutils,
+;; setuptools, easy_install and pip. All of these are sharing the file
+;; setup.py, introduced with distutils in Python 2.0. The setup.py file can be
+;; considered as a kind of Makefile accepting targets (or commands) like
+;; "build" and "install". As of autumn 2016 the recommended way to install
+;; Python packages is using pip.
+;; For both distutils and setuptools, running "python setup.py install" is the
+;; way to install Python packages. With distutils the "install" command
+;; basically copies all packages into <prefix>/lib/pythonX.Y/site-packages.
+;; Some time later "setuptools" was established to enhance distutils. To use
+;; setuptools, the developer imports setuptools in setup.py. When importing
+;; setuptools, the original "install" command gets overwritten by setuptools'
+;; "install" command.
+;; The command-line tools easy_install and pip are both capable of finding and
+;; downloading the package source from PyPI (the Python Package Index). Both
+;; of them import setuptools and execute the "setup.py" file under their
+;; control. Thus the "setup.py" behaves as if the developer had imported
+;; setuptools within setup.py - even is still using only distutils.
+;; Setuptools' "install" command (to be more precise: the "easy_install"
+;; command which is called by "install") will put the path of the currently
+;; installed version of each package and it's dependencies (as declared in
+;; setup.py) into an "easy-install.pth" file. In Guix each packages gets its
+;; own "site-packages" directory and thus an "easy-install.pth" of its own.
+;; To avoid conflicts, the python build system renames the file to
+;; <packagename>.pth in the phase rename-pth-file. To ensure that Python will
+;; process the .pth file, easy_install also creates a basic "site.py" in each
+;; "site-packages" directory. The file is the same for all packages, thus
+;; there is no need to rename it. For more information about .pth files and
+;; the site module, please refere to
+;; The .pth files contain the file-system paths (pointing to the store) of all
+;; dependencies. So the dependency is hidden in the .pth file but is not
+;; visible in the file-system. Now if packages A and B both required packages
+;; P, but in different versions, Guix will not detect this when installing
+;; both A and B to a profile. (For details and example see
+;; Pip behaves a bit different then easy_install: it always executes
+;; "setup.py" with the option "--single-version-externally-managed" set. This
+;; makes setuptools' "install" command run the original "install" command
+;; instead of the "easy_install" command, so no .pth file (and no site.py)
+;; will be created. The "site-packages" directory only contains the package
+;; and the related .egg-info directory.
+;; This is exactly what we need for Guix and this is what we mimic in the
+;; install phase below.
+;; As a draw back, the magic of the .pth file of linking to the other required
+;; packages is gone and these packages have now to be declared as
+;; Note: Importing setuptools also adds two sub-commands: "install_egg_info"
+;; and "install_scripts". These sub-commands are executed even if
+;; "--single-version-externally-managed" is set, thus the .egg-info directory
+;; and the scripts defined in entry-points will always be created.
;; Run setup.py with "setuptools" being imported, which will patch
@@ -149,6 +212,9 @@ when running checks after installing the package."
(define* (rename-pth-file #:key name inputs outputs #:allow-other-keys)
"Rename easy-install.pth to NAME.pth to avoid conflicts between packages
installed with setuptools."
+ ;; Even if the "easy-install.pth" is not longer created, we kept this phase.
+ ;; There still may be packages creating an "easy-install.pth" manually for
+ ;; some good reason.
(let* ((out (assoc-ref outputs "out"))
(python (assoc-ref inputs "python"))
(site-packages (string-append out "/lib/python"
- branch python-build-system created (now d464917), Hartmut Goebel, 2016/11/15
- 01/104: guix: python-build-system: Fix an outdated comment., Hartmut Goebel, 2016/11/15
- 04/104: guix: python-build-system: Import setuptools before calling `setup.py'., Hartmut Goebel, 2016/11/15
- 06/104: guix: python-build-system: Add helpers for getting and setting PYTHONPATH., Hartmut Goebel, 2016/11/15
- 02/104: gnu: ensure pip and setuptools are installed even for Python 2., Hartmut Goebel, 2016/11/15
- 07/104: guix: python-build-system: Delete .egg-info file created in phase check., Hartmut Goebel, 2016/11/15
- 05/104: guix: python-build-system: Add option "#:use-setuptools?" (default true)., Hartmut Goebel, 2016/11/15
- 03/104: guix: build all Python packages with --single-version-externally-managed., Hartmut Goebel, 2016/11/15
- 11/104: lint: more packages to probably be a native input., Hartmut Goebel, 2016/11/15
- 18/104: gnu: Remove needless inputs python-pip and python2-pip., Hartmut Goebel, 2016/11/15
- 08/104: guix: python-build-system: Add background about Python installation methods.,
Hartmut Goebel <=
- 09/104: gnu: python-2.7: Add all guix prefixes in PYTHONPATH to site-prefixes., Hartmut Goebel, 2016/11/15
- 22/104: gnu: Fix python inputs, part 3: all native-inputs become propagated-inputs., Hartmut Goebel, 2016/11/15
- 15/104: gnu: Remove python-setuptools and python2-setuptools from inputs (part 3), Hartmut Goebel, 2016/11/15
- 10/104: guix: Add lint-checker for packages which should be no inputs at all., Hartmut Goebel, 2016/11/15
- 29/104: gnu: python-ccm: Add missing input python-psutil., Hartmut Goebel, 2016/11/15
- 27/104: gnu: python-pytest-cov: Use upstream options for testing., Hartmut Goebel, 2016/11/15
- 21/104: gnu: Fix python inputs, part 2: all inputs become native-inputs., Hartmut Goebel, 2016/11/15
- 33/104: gnu: python-fixture: Correct inputs., Hartmut Goebel, 2016/11/15
- 26/104: gnu: scons: Do not use setuptools for building., Hartmut Goebel, 2016/11/15
- 30/104: gnu: python-ccm: Update synopsis and description., Hartmut Goebel, 2016/11/15