[bug#61701] [PATCH] doc: Propose new cookbook section for reproducible research.

[Simon Tournier]

Hi,

Cool!  Nice!  My suggestions could be dropped, I mean they are just
suggestions. :-)


On Wed, 22 Feb 2023 at 05:17, kyle <kyle@posteo.net> wrote:

> +Writing a manifest for packaging R code alone requires only minimal
> +knowledge of the Guix infrastructure. This stub should work for most
> +cases involving the R packages already in Guix.
> +
> +@example
> +(use-modules
> + (gnu packages cran)
> + (gnu packages statistics))
> +
> +(packages->manifest
> + (list r r-tidyverse))

Missing @end example, no?

> +R packages are defined predominantly inside of gnu/packages/cran.scm and
> +gnu/packages/statistics.scm files under a guix source repository.

When presenting Guix, usually I start with:

    guix shell r r-tidyverse

then

    guix shell r r-tidyverse --export-manifest

--8<---------------cut here---------------start------------->8---
;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.

(specifications->manifest
  (list "r" "r-tidyverse"))
--8<---------------cut here---------------end--------------->8---

which demystifies the “manifest”.  Then, I go via:

    guix show r

to point the location and explains why (gnu packages statistics).  So
then, people connect the dots between the command line invocation and
the manifest:

--8<---------------cut here---------------start------------->8---
(use-modules
 (gnu packages cran)
 (gnu packages statistics))

(packages->manifest
 (list r r-tidyverse))
--8<---------------cut here---------------end--------------->8---

Based on that, I introduce modules and why (guix licenses) for
examples.

> +This manifest can be run with the basic guix shell command:
> +
> +@example
> +guix shell --manifest=manifest.scm --container
> +@end example

Do you need ’container’?  If yes, I should set the scenario before: We
want to run X and Y inside a container, because we want to be sure that
we described all we need to run the analysis.  Well, something like
that.

> +Please remember at the end to pin your channels so that others in the
> +future know how to recover your exact Guix environment.
> +
> +@example
> +guix describe --format=channels > channels.scm
> +@end example
> +
> +This can be done with Guix time machine:
> +
> +@example
> +guix time-machine --channels=channels.scm \
> +  -- guix shell --manifest=manifest.scm --container
> +@end example

No, the correct is:

    guix time-machine --channels=channels.scm \
      -- shell --manifest=manifest.scm --container

without the extra ’guix’.


> +In contrast, the python scientific ecosystem is far less
> +standardized. There is no effort made to integrate all python packages
> +together. While there is a latest python version, it is less often less
> +dominantly used for various reasons such as the fact that python tends
> +to be employed with much larger teams than R is. This makes packaging up
> +reproducible python environments much more difficult. Adding R together
> +with python as a mixture complicates things still further. However, we
> +have to be mindful of the goals of reproducible research.
> +
> +If reproducibility becomes an end in itself and not a catlyst towards
> +faster discovery, then Guix will be a non-starter for scientists. Their
> +goal is to develop useful understanding about particular aspects of the
> +world.

Cool paragraph!

> +Thankfully, three common scenarios cover the vast majority of
> +needs. These are:
> +
> +@itemize
> +@item
> +combining standard package definitions with custom package definitions
> +@item
> +combining package definitions from the current revision with other revisions
> +@item
> +combining package variants which need a modified build-system
> +@end itemize
> +
> +In the rest of the tutorial we develop a manifest which tackles all
> +three of these common issues. The hope is that if you see the hardest
> +possible common situation as being readily solvable without writing
> +thousands of lines of code, researchers will clearly see it as worth the
> +effort which will not pose a significant detour from the main line of
> +their research.
> +
> +@example
> +(use-modules
> + (guix packages)
> + (guix download)
> + (guix licenses)
> + (guix profiles)

Before jumping to that, I would add a paragraph after the first manifest
explaining that other modules can be used, for instance the module (guix
licenses) provides all the licenses known by Guix.  Well, something like
that.

> + (gnu packages)
> + (gnu packages cran)
> + (guix inferior)
> + (guix channels)
> + (guix build-system python))
> +
> +;; guix import pypi APTED

I would add a sentence before this manifest point to the documentation
of “guix import” and explaining with one sentence that it allows to
extract information from PyPI and generates a recipe for Guix.

> +(define python-apted
> + (package
> +  (name "python-apted")
> +  (version "1.0.3")
> +  (source (origin
> +            (method url-fetch)
> +            (uri (pypi-uri "apted" version))
> +            (sha256
> +             (base32
> +              "1sawf6s5c64fgnliwy5w5yxliq2fc215m6alisl7yiflwa0m3ymy"))))
> +  (build-system python-build-system)
> +  (home-page "https://github.com/JoaoFelipe/apted";)
> +  (synopsis "APTED algorithm for the Tree Edit Distance")
> +  (description "APTED algorithm for the Tree Edit Distance")
> +  (license expat)))
> +
> +(define last-guix-with-python-3.6
> + (list
> +  (channel
> +   (name 'guix)
> +   (url "https://git.savannah.gnu.org/git/guix.git";)
> +   (commit
> +    "d66146073def03d1a3d61607bc6b77997284904b"))))
> +
> +(define connection-to-last-guix-with-python-3.6
> + (inferior-for-channels last-guix-with-python-3.6))

Why do you need an inferior?  Is it to avoid the “guix time-machine”?
Ah, no the answer below. :-)

> +(define first car)
> +
> +(define python-3.6
> + (first
> +  (lookup-inferior-packages
> +   connection-to-last-guix-with-python-3.6 "python")))
> +
> +(define python3.6-numpy
> + (first
> +  (lookup-inferior-packages
> +   connection-to-last-guix-with-python-3.6 "python-numpy")))
> +
> +(define included-packages
> + (list r r-reticulate))
> + 
> +(define inferior-packages
> + (list python-3.6 python3.6-numpy))
> +
> +(define package-with-python-3.6
> + (package-with-explicit-python python-3.6
> +  "python-" "python3.6-" 'python3-variant))
> + 
> +(define custom-variant-packages
> + (list (package-with-python-3.6 python-apted)))
> +
> +(concatenate-manifest
> + (map packages->manifest
> +  (list
> +   included-packages
> +   inferior-packages
> +   custom-variant-packages)))

While this is cool, I would not recommend it as some practise.  This
kind of mix can lead to various annoyances, IMHO.  First, it will scale
poorly if you add more inferiors.  Second, the probability that the
resulting computational environment works well decreases.

Anyway, the example is cool! :-)

> +@end example
> +
> +This should produce a profile with the latest R and an older python
> +3.6. These should be able to interoperate with code like:
> +
> +@example
> +library(reticulate)
> +use_python("python")
> +apted = import("apted")
> +t1 = '{a{b}{c}}'
> +t2 = '{a{b{d}}}'
> +metric = apted$APTED(t1, t2)
> +distance = metric$compute_edit_distance()
> +@end example

This example is cool!


> diff --git a/guix/build-system/python.scm b/guix/build-system/python.scm
> index c8f04b2298..d4aaab906d 100644
> --- a/guix/build-system/python.scm
> +++ b/guix/build-system/python.scm
> @@ -36,6 +36,7 @@ (define-module (guix build-system python)
>    #:use-module (srfi srfi-1)
>    #:use-module (srfi srfi-26)
>    #:export (%python-build-system-modules
> +            package-with-explicit-python
>              package-with-python2
>              strip-python2-variant
>              default-python

Maybe this could be a separated patch.


Cheers,
simon