[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH] Add Bioconductor importer and updater.
|
From: |
Ricardo Wurmus |
|
Subject: |
[PATCH] Add Bioconductor importer and updater. |
|
Date: |
Thu, 17 Dec 2015 12:34:05 +0100 |
Hi Guix,
Bioconductor is a large, popular repository of R packages for
bioinformatics. It has coordinated, versioned releases (latest stable
release is 3.2).
The attached patches generalise the existing procedures used by the CRAN
importer, so that with only little effort importers for other R
repositories are possible. The last patch adds an importer for the
Bioconductor repository.
Imported package expressions now also contain a package property
“r-repository”, which is either the symbol “cran” or the symbol
“bioconductor”. This is also checked in the predicates “cran-package?”
and “bioconductor-package?” used by the updaters.
The first two patches in this series are actually unrelated: the first
fixes an annoying bug in the CRAN importer; the second corrects an
outdated claim in the CRAN importer’s documentation.
~~ Ricardo
>From 770a92ed51641ea2da7893c4d70206fd19f0513b Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <address@hidden>
Date: Wed, 16 Dec 2015 14:29:38 +0100
Subject: [PATCH 1/6] import: Drop empty list items.
* guix/import/cran.scm (listify): Remove empty strings from result list.
---
guix/import/cran.scm | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 845ecb5..45c679c 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -128,9 +128,12 @@ empty list when the FIELD cannot be found."
#f "( *\\([^\\)]+\\)) *"
value 'pre 'post)
#\,)))
- ;; When there is whitespace inside of items it is probably because
- ;; this was not an actual list to begin with.
- (remove (cut string-any char-set:whitespace <>)
+ (remove (lambda (item)
+ (or (string-null? item)
+ ;; When there is whitespace inside of items it is
+ ;; probably because this was not an actual list to
+ ;; begin with.
+ (string-any char-set:whitespace item)))
(map string-trim-both items))))))
(define (beautify-description description)
--
2.1.0
>From ee6f9d5855528870cf861f5cdd07dd6e654f3736 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <address@hidden>
Date: Thu, 17 Dec 2015 12:23:35 +0100
Subject: [PATCH 2/6] doc: Fix outdated claim about CRAN importer.
* doc/guix.texi: The CRAN importer no longer extracts information from
the HTML package description but from the package's DESCRIPTION file.
---
doc/guix.texi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index f63e366..05a94dc 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -4205,7 +4205,7 @@ Import meta-data from @uref{http://cran.r-project.org/,
CRAN}, the
central repository for the @uref{http://r-project.org, address@hidden
statistical and graphical environment}.
-Information is extracted from the HTML package description.
+Information is extracted from the package's DESCRIPTION file.
The command command below imports meta-data for the @code{Cairo}
R package:
--
2.1.0
>From 9fd0e0fea086a07a1219fb2bca9cb739d4373251 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <address@hidden>
Date: Wed, 16 Dec 2015 14:02:29 +0100
Subject: [PATCH 3/6] build: Add bioconductor-uri procedure.
* guix/build-system/r.scm (bioconductor-uri): New procedure.
---
guix/build-system/r.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/guix/build-system/r.scm b/guix/build-system/r.scm
index da06cb1..a8ca354 100644
--- a/guix/build-system/r.scm
+++ b/guix/build-system/r.scm
@@ -29,7 +29,8 @@
#:export (%r-build-system-modules
r-build
r-build-system
- cran-uri))
+ cran-uri
+ bioconductor-uri))
;; Commentary:
;;
@@ -46,6 +47,12 @@ available via the first URI, the second URI points to the
archived version."
(string-append "mirror://cran/src/contrib/Archive/"
name "/" name "_" version ".tar.gz")))
+(define (bioconductor-uri name version)
+ "Return a URI string for the R package archive on Bioconductor for the
+release corresponding to NAME and VERSION."
+ (string-append "http://bioconductor.org/packages/release/bioc/src/contrib/";
+ name "_" version ".tar.gz"))
+
(define %r-build-system-modules
;; Build-side modules imported by default.
`((guix build r-build-system)
--
2.1.0
>From 04b9cb226f90b772f24a766fb632d8238e211366 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <address@hidden>
Date: Wed, 16 Dec 2015 14:22:17 +0100
Subject: [PATCH 4/6] import: Add package->upstream-name procedure.
* guix/import/cran.scm (package->upstream-name): New procedure.
---
guix/import/cran.scm | 40 +++++++++++++++++++++++-----------------
1 file changed, 23 insertions(+), 17 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 45c679c..bf46d17 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -204,27 +204,33 @@ which was derived from the R package's DESCRIPTION file."
;;; Updater.
;;;
+(define (package->upstream-name package)
+ "Return the upstream name of the PACKAGE."
+ (let* ((properties (package-properties package))
+ (upstream-name (and=> properties
+ (cut assoc-ref <> 'upstream-name))))
+ (if upstream-name
+ upstream-name
+ (match (package-source package)
+ ((? origin? origin)
+ (match (origin-uri origin)
+ ((url rest ...)
+ (let ((end (string-rindex url #\_))
+ (start (string-rindex url #\/)))
+ ;; The URL ends on
+ ;; (string-append "/" name "_" version ".tar.gz")
+ (substring url start end)))
+ (_ #f)))
+ (_ #f)))))
+
(define (latest-release package)
"Return an <upstream-source> for the latest release of PACKAGE."
- (define (package->cran-name package)
- (match (package-source package)
- ((? origin? origin)
- (match (origin-uri origin)
- ((url rest ...)
- (let ((end (string-rindex url #\_))
- (start (string-rindex url #\/)))
- ;; The URL ends on
- ;; (string-append "/" name "_" version ".tar.gz")
- (substring url start end)))
- (_ #f)))
- (_ #f)))
-
- (define cran-name
- (package->cran-name (specification->package package)))
+ (define upstream-name
+ (package->upstream-name (specification->package package)))
(define meta
- (cran-fetch cran-name))
+ (cran-fetch upstream-name))
(and meta
(let ((version (assoc-ref meta "Version")))
@@ -232,7 +238,7 @@ which was derived from the R package's DESCRIPTION file."
(upstream-source
(package package)
(version version)
- (urls (cran-uri cran-name version))))))
+ (urls (cran-uri upstream-name version))))))
(define (cran-package? package)
"Return true if PACKAGE is an R package from CRAN."
--
2.1.0
>From 79a8349296fb46d24caf3b4f35ae347b0f5aadee Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <address@hidden>
Date: Wed, 16 Dec 2015 14:28:43 +0100
Subject: [PATCH 5/6] import: Rename "cran-fetch" to "fetch-description".
* guix/import/cran.scm (cran-fetch): Rename procedure ...
(fetch-description): ... to this.
---
guix/import/cran.scm | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index bf46d17..fc27090 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -109,11 +109,11 @@ package definition."
(define %cran-url "http://cran.r-project.org/web/packages/";)
-(define (cran-fetch name)
+(define (fetch-description base-url name)
"Return an alist of the contents of the DESCRIPTION file for the R package
NAME, or #f on failure. NAME is case-sensitive."
;; This API always returns the latest release of the module.
- (let ((url (string-append %cran-url name "/DESCRIPTION")))
+ (let ((url (string-append base-url name "/DESCRIPTION")))
(description->alist (read-string (http-fetch url)))))
(define (listify meta field)
@@ -196,7 +196,7 @@ which was derived from the R package's DESCRIPTION file."
(define (cran->guix-package package-name)
"Fetch the metadata for PACKAGE-NAME from cran.r-project.org, and return the
`package' s-expression corresponding to that package, or #f on failure."
- (let ((module-meta (cran-fetch package-name)))
+ (let ((module-meta (fetch-description %cran-url package-name)))
(and=> module-meta description->package)))
@@ -230,7 +230,7 @@ which was derived from the R package's DESCRIPTION file."
(package->upstream-name (specification->package package)))
(define meta
- (cran-fetch upstream-name))
+ (fetch-description %cran-url upstream-name))
(and meta
(let ((version (assoc-ref meta "Version")))
--
2.1.0
>From 3af383f041762778cb61d48f5e00b9656d616704 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <address@hidden>
Date: Wed, 16 Dec 2015 14:45:28 +0100
Subject: [PATCH 6/6] import: Add Bioconductor importer and updater.
* guix/import/cran.scm (bioconductor->guix-package,
%bioconductor-updater, latest-bioconductor-release,
bioconductor-package?): New procedures.
(%bioconductor-url, %bioconductor-svn-url): New variables.
(description->package): Update signature to distinguish between packages
from different repositories.
(latest-release): Rename procedure ...
(latest-cran-release): ... to this.
(cran-package?): Do not assume all R packages are available on CRAN.
* tests/cran.scm: Update tests.
* guix/scripts/import.scm (importers): Add "bioconductor" importers.
* guix/scripts/refresh.scm (%updaters): Add "%bioconductor-updater".
* doc/guix.texi: Document Bioconductor importer and updater.
---
doc/guix.texi | 18 ++++++++++
guix/import/cran.scm | 93 +++++++++++++++++++++++++++++++++++++++---------
guix/scripts/import.scm | 3 +-
guix/scripts/refresh.scm | 1 +
tests/cran.scm | 5 +--
5 files changed, 101 insertions(+), 19 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 05a94dc..44f9daf 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -4214,6 +4214,22 @@ R package:
guix import cran Cairo
@end example
address@hidden bioconductor
address@hidden Bioconductor
+Import meta-data from @uref{http://www.bioconductor.org/, Bioconductor},
+a repository of R packages for for the analysis and comprehension of
+high-throughput genomic data in bioinformatics.
+
+Information is extracted from a package's DESCRIPTION file published on
+the web interface of the Bioconductor SVN repository.
+
+The command command below imports meta-data for the @code{GenomicRanges}
+R package:
+
address@hidden
+guix import bioconductor GenomicRanges
address@hidden example
+
@item nix
Import meta-data from a local copy of the source of the
@uref{http://nixos.org/nixpkgs/, Nixpkgs address@hidden
@@ -4412,6 +4428,8 @@ the updater for GNOME packages;
the updater for @uref{http://elpa.gnu.org/, ELPA} packages;
@item cran
the updater for @uref{http://cran.r-project.org/, CRAN} packages;
address@hidden bioconductor
+the updater for @uref{http://www.bioconductor.org/, Bioconductor} packages;
@item pypi
the updater for @uref{https://pypi.python.org, PyPI} packages.
@end table
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index fc27090..35b18b1 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -29,12 +29,14 @@
#:use-module (guix base32)
#:use-module ((guix download) #:select (download-to-store))
#:use-module (guix import utils)
- #:use-module ((guix build-system r) #:select (cran-uri))
+ #:use-module ((guix build-system r) #:select (cran-uri bioconductor-uri))
#:use-module (guix upstream)
#:use-module (guix packages)
#:use-module (gnu packages)
#:export (cran->guix-package
- %cran-updater))
+ bioconductor->guix-package
+ %cran-updater
+ %bioconductor-updater))
;;; Commentary:
;;;
@@ -108,6 +110,15 @@ package definition."
`((,type (,'quasiquote ,(format-inputs package-inputs)))))))
(define %cran-url "http://cran.r-project.org/web/packages/";)
+(define %bioconductor-url "http://bioconductor.org/packages/";)
+
+;; The latest Bioconductor release is 3.2. Bioconductor packages should be
+;; updated together.
+(define %bioconductor-svn-url
+ (string-append "https://readonly:readonly@";
+ "hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_2/"
+ "madman/Rpacks/"))
+
(define (fetch-description base-url name)
"Return an alist of the contents of the DESCRIPTION file for the R package
@@ -147,24 +158,31 @@ into a proper sentence and by using two spaces between
sentences."
(regexp-substitute/global #f "\\. \\b"
cleaned 'pre ". " 'post)))
-(define (description->package meta)
- "Return the `package' s-expression for a CRAN package from the alist META,
-which was derived from the R package's DESCRIPTION file."
+(define (description->package repository meta)
+ "Return the `package' s-expression for an R package published on REPOSITORY
+from the alist META, which was derived from the R package's DESCRIPTION file."
(define (guix-name name)
(if (string-prefix? "r-" name)
(string-downcase name)
(string-append "r-" (string-downcase name))))
- (let* ((name (assoc-ref meta "Package"))
+ (let* ((base-url (case repository
+ ((cran) %cran-url)
+ ((bioconductor) %bioconductor-url)))
+ (uri-helper (case repository
+ ((cran) cran-uri)
+ ((bioconductor) bioconductor-uri)))
+ (name (assoc-ref meta "Package"))
(synopsis (assoc-ref meta "Title"))
(version (assoc-ref meta "Version"))
(license (string->license (assoc-ref meta "License")))
;; Some packages have multiple home pages. Some have none.
(home-page (match (listify meta "URL")
((url rest ...) url)
- (_ (string-append %cran-url name))))
- (source-url (match (cran-uri name version)
+ (_ (string-append base-url name))))
+ (source-url (match (uri-helper name version)
((url rest ...) url)
+ ((? string? url) url)
(_ #f)))
(tarball (with-store store (download-to-store store source-url)))
(sysdepends (map string-downcase (listify meta "SystemRequirements")))
@@ -178,16 +196,17 @@ which was derived from the R package's DESCRIPTION file."
(version ,version)
(source (origin
(method url-fetch)
- (uri (cran-uri ,name version))
+ (uri (,(procedure-name uri-helper) ,name version))
(sha256
(base32
,(bytevector->nix-base32-string (file-sha256 tarball))))))
- (properties ,`(,'quasiquote ((,'upstream-name . ,name))))
+ (properties ,`(,'quasiquote ((,'upstream-name . ,name)
+ (,'r-repository . ,repository))))
(build-system r-build-system)
,@(maybe-inputs sysdepends)
,@(maybe-inputs propagate 'propagated-inputs)
(home-page ,(if (string-null? home-page)
- (string-append %cran-url name)
+ (string-append base-url name)
home-page))
(synopsis ,synopsis)
(description ,(beautify-description (assoc-ref meta "Description")))
@@ -197,7 +216,13 @@ which was derived from the R package's DESCRIPTION file."
"Fetch the metadata for PACKAGE-NAME from cran.r-project.org, and return the
`package' s-expression corresponding to that package, or #f on failure."
(let ((module-meta (fetch-description %cran-url package-name)))
- (and=> module-meta description->package)))
+ (and=> module-meta (cut description->package 'cran <>))))
+
+(define (bioconductor->guix-package package-name)
+ "Fetch the metadata for PACKAGE-NAME from bioconductor.org, and return the
+`package' s-expression corresponding to that package, or #f on failure."
+ (let ((module-meta (fetch-description %bioconductor-svn-url package-name)))
+ (and=> module-meta (cut description->package 'bioconductor <>))))
;;;
@@ -223,7 +248,7 @@ which was derived from the R package's DESCRIPTION file."
(_ #f)))
(_ #f)))))
-(define (latest-release package)
+(define (latest-cran-release package)
"Return an <upstream-source> for the latest release of PACKAGE."
(define upstream-name
@@ -240,16 +265,52 @@ which was derived from the R package's DESCRIPTION file."
(version version)
(urls (cran-uri upstream-name version))))))
+(define (latest-bioconductor-release package)
+ "Return an <upstream-source> for the latest release of PACKAGE."
+
+ (define upstream-name
+ (package->upstream-name (specification->package package)))
+
+ (define meta
+ (fetch-description %bioconductor-svn-url upstream-name))
+
+ (and meta
+ (let ((version (assoc-ref meta "Version")))
+ ;; Bioconductor does not provide signatures.
+ (upstream-source
+ (package package)
+ (version version)
+ (urls (bioconductor-uri upstream-name version))))))
+
(define (cran-package? package)
"Return true if PACKAGE is an R package from CRAN."
- ;; Assume all R packages are available on CRAN.
- (string-prefix? "r-" (package-name package)))
+ ;; Assume all R packages are available on CRAN, unless otherwise indicated
+ ;; by the r-repository property.
+ (let ((properties (package-properties package)))
+ (and (string-prefix? "r-" (package-name package))
+ (or (not properties)
+ (not (assoc-ref properties 'r-repository))
+ (eqv? 'cran (assoc-ref properties 'r-repository))))))
+
+(define (bioconductor-package? package)
+ "Return true if PACKAGE is an R package from Bioconductor."
+ (let ((properties (package-properties package)))
+ (and (string-prefix? "r-" (package-name package))
+ properties
+ (eqv? 'bioconductor (assoc-ref properties 'r-repository)))))
(define %cran-updater
(upstream-updater
(name 'cran)
(description "Updater for CRAN packages")
(pred cran-package?)
- (latest latest-release)))
+ (latest latest-cran-release)))
+
+(define %bioconductor-updater
+ (upstream-updater
+ (name 'bioconductor)
+ (description "Updater for Bioconductor packages")
+ (pred bioconductor-package?)
+ (latest latest-bioconductor-release)))
;;; cran.scm ends here
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 7b29794..5810ef8 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -73,7 +73,8 @@ rather than \\n."
;;; Entry point.
;;;
-(define importers '("gnu" "nix" "pypi" "cpan" "hackage" "elpa" "gem" "cran"))
+(define importers '("gnu" "nix" "pypi" "cpan" "hackage" "elpa" "gem" "cran"
+ "bioconductor"))
(define (resolve-importer name)
(let ((module (resolve-interface
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index a5834d1..f9e3f31 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -195,6 +195,7 @@ unavailable optional dependencies such as Guile-JSON."
%gnome-updater
%elpa-updater
%cran-updater
+ %bioconductor-updater
((guix import pypi) => %pypi-updater)))
(define (lookup-updater name)
diff --git a/tests/cran.scm b/tests/cran.scm
index 0a4a2fd..72df2b3 100644
--- a/tests/cran.scm
+++ b/tests/cran.scm
@@ -107,7 +107,7 @@ Date/Publication: 2015-07-14 14:15:16
("mirror://cran/src/contrib/My-Example_1.2.3.tar.gz"
"source")
(_ (error "Unexpected URL: " url))))))))
- (match ((@@ (guix import cran) description->package) description-alist)
+ (match ((@@ (guix import cran) description->package) 'cran
description-alist)
(('package
('name "r-my-example")
('version "1.2.3")
@@ -117,7 +117,8 @@ Date/Publication: 2015-07-14 14:15:16
('sha256
('base32
(? string? hash)))))
- ('properties ('quasiquote (('upstream-name . "My-Example"))))
+ ('properties ('quasiquote (('upstream-name . "My-Example")
+ ('r-repository . 'cran))))
('build-system 'r-build-system)
('inputs
('quasiquote
--
2.1.0
- [PATCH] Add Bioconductor importer and updater.,
Ricardo Wurmus <=