[bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl).

[Ludovic Courtès]

* guix/inferior.scm, guix/scripts/repl.scm, tests/inferior.scm: New
files.
* Makefile.am (MODULES): Add 'guix/scripts/repl.scm' and
'guix/inferior.scm'.
(SCM_TESTS): Add 'tests/inferior.scm'.
* doc/guix.texi (Invoking guix repl): New node.
---
 Makefile.am           |   3 +
 doc/guix.texi         |  53 +++++++++++
 guix/inferior.scm     | 197 +++++++++++++++++++++++++++++++++++++++++
 guix/scripts/repl.scm | 199 ++++++++++++++++++++++++++++++++++++++++++
 tests/inferior.scm    |  69 +++++++++++++++
 5 files changed, 521 insertions(+)
 create mode 100644 guix/inferior.scm
 create mode 100644 guix/scripts/repl.scm
 create mode 100644 tests/inferior.scm

diff --git a/Makefile.am b/Makefile.am
index 5dc04de35..8d05744bc 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -85,6 +85,7 @@ MODULES =                                     \
   guix/nar.scm                                 \
   guix/derivations.scm                         \
   guix/grafts.scm                              \
+  guix/inferior.scm                            \
   guix/gnu-maintenance.scm                     \
   guix/self.scm                                        \
   guix/upstream.scm                            \
@@ -200,6 +201,7 @@ MODULES =                                   \
   guix/scripts/substitute.scm                  \
   guix/scripts/authenticate.scm                        \
   guix/scripts/refresh.scm                     \
+  guix/scripts/repl.scm                                \
   guix/scripts/system.scm                      \
   guix/scripts/system/search.scm               \
   guix/scripts/lint.scm                                \
@@ -355,6 +357,7 @@ SCM_TESTS =                                 \
   tests/profiles.scm                           \
   tests/search-paths.scm                       \
   tests/syscalls.scm                           \
+  tests/inferior.scm                           \
   tests/gremlin.scm                            \
   tests/bournish.scm                           \
   tests/lint.scm                               \
diff --git a/doc/guix.texi b/doc/guix.texi
index 7ce364b0a..e93b320e8 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -166,6 +166,7 @@ Programming Interface
 * Derivations::                 Low-level interface to package derivations.
 * The Store Monad::             Purely functional interface to the store.
 * G-Expressions::               Manipulating build expressions.
+* Invoking guix repl::          Fiddling with Guix interactively.
 
 Defining Packages
 
@@ -3267,6 +3268,7 @@ package definitions.
 * Derivations::                 Low-level interface to package derivations.
 * The Store Monad::             Purely functional interface to the store.
 * G-Expressions::               Manipulating build expressions.
+* Invoking guix repl::          Fiddling with Guix interactively.
 @end menu
 
 @node Defining Packages
@@ -5538,6 +5540,57 @@ corresponding to @var{obj} for @var{system}, 
cross-compiling for
 has an associated gexp compiler, such as a @code{<package>}.
 @end deffn
 
address@hidden Invoking guix repl
address@hidden Invoking @command{guix repl}
+
address@hidden REPL, read-eval-print loop
+The @command{guix repl} command spawns a Guile @dfn{read-eval-print loop}
+(REPL) for interactive programming (@pxref{Using Guile Interactively,,, guile,
+GNU Guile Reference Manual}).  Compared to just launching the @command{guile}
+command, @command{guix repl} guarantees that all the Guix modules and all its
+dependencies are available in the search path.  You can use it this way:
+
address@hidden
+$ guix repl
+scheme@@(guile-user)> ,use (gnu packages base)
+scheme@@(guile-user)> coreutils
+$1 = #<package coreutils@@8.29 gnu/packages/base.scm:327 3e28300>
address@hidden example
+
address@hidden inferiors
+In addition, @command{guix repl} implements a simple machine-readable REPL
+protocol for use by @code{(guix inferior)}, a facility to interact with
address@hidden, separate processes running a potentially different revision
+of Guix.
+
+The available options are as follows:
+
address@hidden @code
address@hidden address@hidden
address@hidden -t @var{type}
+Start a REPL of the given @var{TYPE}, which can be one of the following:
+
address@hidden @code
address@hidden guile
+This is default, and it spawns a standard full-featured Guile REPL.
address@hidden machine
+Spawn a REPL that uses the machine-readable protocol.  This is the protocol
+that the @code{(guix inferior)} module speaks.
address@hidden table
+
address@hidden address@hidden
+By default, @command{guix repl} reads from standard input and writes to
+standard output.  When this option is passed, it will instead listen for
+connections on @var{endpoint}.  Here are examples of valid options:
+
address@hidden @code
address@hidden --listen=tcp:37146
+Accept connections on localhost on port 37146.
+
address@hidden --listen=unix:/tmp/socket
+Accept connections on the Unix-domain socket @file{/tmp/socket}.
address@hidden table
address@hidden table
 
 @c *********************************************************************
 @node Utilities
diff --git a/guix/inferior.scm b/guix/inferior.scm
new file mode 100644
index 000000000..629c2c431
--- /dev/null
+++ b/guix/inferior.scm
@@ -0,0 +1,197 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix inferior)
+  #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 popen)
+  #:export (inferior?
+            open-inferior
+            close-inferior
+            inferior-eval
+            inferior-object?
+
+            inferior-package?
+            inferior-package-name
+            inferior-package-version
+
+            inferior-packages
+            inferior-package-synopsis
+            inferior-package-description))
+
+;;; Commentary:
+;;;
+;;; This module provides a way to spawn Guix "inferior" processes and to talk
+;;; to them.  It allows us, from one instance of Guix, to interact with
+;;; another instance of Guix coming from a different commit.
+;;;
+;;; Code:
+
+;; Inferior Guix process.
+(define-record-type <inferior>
+  (inferior pid socket version)
+  inferior?
+  (pid      inferior-pid)
+  (socket   inferior-socket)
+  (version  inferior-version))                    ;REPL protocol version
+
+(define (inferior-pipe directory command)
+  "Return an input/output pipe on the Guix instance in DIRECTORY.  This runs
+'DIRECTORY/COMMAND repl' if it exists, or falls back to some other method if
+it's an old Guix."
+  (let ((pipe (with-error-to-port (%make-void-port "w")
+                (lambda ()
+                  (open-pipe* OPEN_BOTH
+                              (string-append directory "/" command)
+                              "repl" "-t" "machine")))))
+    (if (eof-object? (peek-char pipe))
+        (begin
+          (close-pipe pipe)
+
+          ;; Older versions of Guix didn't have a 'guix repl' command, so
+          ;; emulate it.
+          (open-pipe* OPEN_BOTH "guile"
+                      "-L" (string-append directory "/share/guile/site/"
+                                          (effective-version))
+                      "-C" (string-append directory "/share/guile/site/"
+                                          (effective-version))
+                      "-C" (string-append directory "/lib/guile/"
+                                          (effective-version) "/site-ccache")
+                      "-c"
+                      (object->string
+                       `(begin
+                          (primitive-load ,(search-path %load-path
+                                                        
"guix/scripts/repl.scm"))
+                          ((@ (guix scripts repl) machine-repl))))))
+        pipe)))
+
+(define* (open-inferior directory #:key (command "bin/guix"))
+  "Open the inferior Guix in DIRECTORY, running 'DIRECTORY/COMMAND repl' or
+equivalent.  Return #f if the inferior could not be launched."
+  (define pipe
+    (inferior-pipe directory command))
+
+  (setvbuf pipe _IOLBF)
+  (match (read pipe)
+    (('repl-version 0 rest ...)
+     (let ((result (inferior 'pipe pipe (cons 0 rest))))
+       (inferior-eval '(use-modules (guix)) result)
+       (inferior-eval '(use-modules (gnu)) result)
+       (inferior-eval '(define %package-table (make-hash-table))
+                      result)
+       result))
+    (_
+     #f)))
+
+(define (close-inferior inferior)
+  "Close INFERIOR."
+  (close-pipe (inferior-socket inferior)))
+
+;; Non-self-quoting object of the inferior.
+(define-record-type <inferior-object>
+  (inferior-object address appearance)
+  inferior-object?
+  (address     inferior-object-address)
+  (appearance  inferior-object-appearance))
+
+(define (write-inferior-object object port)
+  (match object
+    (($ <inferior-object> _ appearance)
+     (format port "#<inferior-object ~a>" appearance))))
+
+(set-record-type-printer! <inferior-object> write-inferior-object)
+
+(define (inferior-eval exp inferior)
+  "Evaluate EXP in INFERIOR."
+  (define sexp->object
+    (match-lambda
+      (('value value)
+       value)
+      (('non-self-quoting address string)
+       (inferior-object address string))))
+
+  (write exp (inferior-socket inferior))
+  (newline (inferior-socket inferior))
+  (match (read (inferior-socket inferior))
+    (('values objects ...)
+     (apply values (map sexp->object objects)))
+    (('exception key objects ...)
+     (apply throw key (map sexp->object objects)))))
+
+
+;;;
+;;; Inferior packages.
+;;;
+
+(define-record-type <inferior-package>
+  (inferior-package inferior name version id)
+  inferior-package?
+  (inferior   inferior-package-inferior)
+  (name       inferior-package-name)
+  (version    inferior-package-version)
+  (id         inferior-package-id))
+
+(define (write-inferior-package package port)
+  (match package
+    (($ <inferior-package> _ name version)
+     (format port "#<inferior-package address@hidden ~a>"
+             name version
+             (number->string (object-address package) 16)))))
+
+(set-record-type-printer! <inferior-package> write-inferior-package)
+
+(define (inferior-packages inferior)
+  "Return the list of packages known to INFERIOR."
+  (let ((result (inferior-eval
+                 '(fold-packages (lambda (package result)
+                                   (let ((id (object-address package)))
+                                     (hashv-set! %package-table id package)
+                                     (cons (list (package-name package)
+                                                 (package-version package)
+                                                 id)
+                                           result)))
+                                 '())
+                 inferior)))
+    (map (match-lambda
+           ((name version id)
+            (inferior-package inferior name version id)))
+         result)))
+
+(define (inferior-package-field package getter)
+  "Return the field of PACKAGE, an inferior package, accessed with GETTER."
+  (let ((inferior (inferior-package-inferior package))
+        (id       (inferior-package-id package)))
+    (inferior-eval `(,getter (hashv-ref %package-table ,id))
+                   inferior)))
+
+(define* (inferior-package-synopsis package #:key (translate? #t))
+  "Return the Texinfo synopsis of PACKAGE, an inferior package.  When
+TRANSLATE? is true, translate it to the current locale's language."
+  (inferior-package-field package
+                          (if translate?
+                              '(compose (@ (guix ui) P_) package-synopsis)
+                              'package-synopsis)))
+
+(define* (inferior-package-description package #:key (translate? #t))
+  "Return the Texinfo description of PACKAGE, an inferior package.  When
+TRANSLATE? is true, translate it to the current locale's language."
+  (inferior-package-field package
+                          (if translate?
+                              '(compose (@ (guix ui) P_) package-description)
+                              'package-description)))
diff --git a/guix/scripts/repl.scm b/guix/scripts/repl.scm
new file mode 100644
index 000000000..b157833a4
--- /dev/null
+++ b/guix/scripts/repl.scm
@@ -0,0 +1,199 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts repl)
+  #:use-module (guix ui)
+  #:use-module (guix scripts)
+  #:use-module (guix utils)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-37)
+  #:use-module (ice-9 match)
+  #:use-module (rnrs bytevectors)
+  #:autoload   (system repl repl) (start-repl)
+  #:autoload   (system repl server)
+                  (make-tcp-server-socket make-unix-domain-server-socket)
+  #:export (machine-repl
+            guix-repl))
+
+;;; Commentary:
+;;;
+;;; This command provides a Guile REPL
+
+(define %default-options
+  `((type . guile)))
+
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix repl")))
+        (option '(#\t "type") #t #f
+                (lambda (opt name arg result)
+                  (alist-cons 'type (string->symbol arg) result)))
+        (option '("listen") #t #f
+                (lambda (opt name arg result)
+                  (alist-cons 'listen arg result)))))
+
+
+(define (show-help)
+  (display (G_ "Usage: guix repl [OPTIONS...]
+Start a Guile REPL in the Guix execution environment.\n"))
+  (display (G_ "
+  -t, --type=TYPE        start a REPL of the given TYPE"))
+  (newline)
+  (display (G_ "
+  -h, --help             display this help and exit"))
+  (display (G_ "
+  -V, --version          display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+(define (self-quoting? x)
+  "Return #t if X is self-quoting."
+  (letrec-syntax ((one-of (syntax-rules ()
+                            ((_) #f)
+                            ((_ pred rest ...)
+                             (or (pred x)
+                                 (one-of rest ...))))))
+    (one-of symbol? string? pair? null? vector?
+            bytevector? number? boolean?)))
+
+(define user-module
+  ;; Module where we execute user code.
+  (let ((module (resolve-module '(guix-user) #f #f #:ensure #t)))
+    (beautify-user-module! module)
+    module))
+
+(define* (machine-repl #:optional
+                       (input (current-input-port))
+                       (output (current-output-port)))
+  "Run a machine-usable REPL over ports INPUT and OUTPUT.
+
+The protocol of this REPL is meant to be machine-readable and provides proper
+support to represent multiple-value returns, exceptions, objects that lack a
+read syntax, and so on.  As such it is more convenient and robust than parsing
+Guile's REPL prompt."
+  (define (value->sexp value)
+    (if (self-quoting? value)
+        `(value ,value)
+        `(non-self-quoting ,(object-address value)
+                           ,(object->string value))))
+
+  (write `(repl-version 0 0) output)
+  (newline output)
+  (force-output output)
+
+  (let loop ()
+    (match (read input)
+      ((? eof-object?) #t)
+      (exp
+       (catch #t
+         (lambda ()
+           (let ((results (call-with-values
+                              (lambda ()
+
+                                (primitive-eval exp))
+                            list)))
+             (write `(values ,@(map value->sexp results))
+                    output)
+             (newline output)
+             (force-output output)))
+         (lambda (key . args)
+           (write `(exception ,key ,@(map value->sexp args)))
+           (newline output)
+           (force-output output)))
+       (loop)))))
+
+(define (call-with-connection spec thunk)
+  "Dynamically-bind the current input and output ports according to SPEC and
+call THUNK."
+  (if (not spec)
+      (thunk)
+
+      ;; Note: the "PROTO:" prefix in SPEC is here so that we can eventually
+      ;; parse things like "fd:123" in a non-ambiguous way.
+      (match (string-index spec #\:)
+        (#f
+         (leave (G_ "~A: invalid listen specification~%") spec))
+        (index
+         (let ((protocol (string-take spec index))
+               (address  (string-drop spec (+ index 1))))
+           (define socket
+             (match protocol
+               ("tcp"
+                (make-tcp-server-socket #:port (string->number address)))
+               ("unix"
+                (make-unix-domain-server-socket #:path address))
+               (_
+                (leave (G_ "~A: unsupported protocol family~%")
+                       protocol))))
+
+           (listen socket 10)
+           (let loop ()
+             (match (accept socket)
+               ((connection . address)
+                (if (= AF_UNIX (sockaddr:fam address))
+                    (info (G_ "accepted connection~%"))
+                    (info (G_ "accepted connection from ~a~%")
+                          (inet-ntop (sockaddr:fam address)
+                                     (sockaddr:addr address))))
+                (dynamic-wind
+                  (const #t)
+                  (lambda ()
+                    (parameterize ((current-input-port connection)
+                                   (current-output-port connection))
+                      (thunk)))
+                  (lambda ()
+                    (false-if-exception (close-port connection))
+                    (info (G_ "connection closed~%"))))))
+             (loop)))))))
+
+
+(define (guix-repl . args)
+  (define opts
+    ;; Return the list of package names.
+    (args-fold* args %options
+                (lambda (opt name arg result)
+                  (leave (G_ "~A: unrecognized option~%") name))
+                (lambda (arg result)
+                  (leave (G_ "~A: extraneous argument~%") arg))
+                %default-options))
+
+  (with-error-handling
+    (let ((type (assoc-ref opts 'type)))
+      (call-with-connection (assoc-ref opts 'listen)
+        (lambda ()
+          (case type
+            ((guile)
+             (save-module-excursion
+              (lambda ()
+                (set-current-module user-module)
+                (start-repl))))
+            ((machine)
+             (machine-repl))
+            (else
+             (leave (G_ "~a: unknown type of REPL~%") type))))))))
+
+;; Local Variables:
+;; eval: (put 'call-with-connection 'scheme-indent-function 1)
+;; End:
diff --git a/tests/inferior.scm b/tests/inferior.scm
new file mode 100644
index 000000000..5e0f8ae66
--- /dev/null
+++ b/tests/inferior.scm
@@ -0,0 +1,69 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-inferior)
+  #:use-module (guix inferior)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-64))
+
+(define %top-srcdir
+  (dirname (search-path %load-path "guix.scm")))
+
+(define %top-builddir
+  (dirname (search-path %load-compiled-path "guix.go")))
+
+
+(test-begin "inferior")
+
+(test-equal "open-inferior"
+  '(42 #t)
+  (let ((inferior (open-inferior %top-builddir
+                                 #:command "scripts/guix")))
+    (and (inferior? inferior)
+         (let ((a (inferior-eval '(apply * '(6 7)) inferior))
+               (b (inferior-eval '(@ (gnu packages base) coreutils)
+                                 inferior)))
+           (close-inferior inferior)
+           (list a (inferior-object? b))))))
+
+(test-equal "inferior-packages"
+  (take (sort (fold-packages (lambda (package lst)
+                               (alist-cons (package-name package)
+                                           (package-version package)
+                                           lst))
+                             '())
+              (lambda (x y)
+                (string<? (car x) (car y))))
+        10)
+  (let* ((inferior (open-inferior %top-builddir
+                                  #:command "scripts/guix"))
+         (packages (inferior-packages inferior)))
+    (and (every string? (map inferior-package-synopsis packages))
+         (begin
+           (close-inferior inferior)
+           (take (sort (map (lambda (package)
+                              (cons (inferior-package-name package)
+                                    (inferior-package-version package)))
+                            packages)
+                       (lambda (x y)
+                         (string<? (car x) (car y))))
+                 10)))))
+
+(test-end "inferior")
-- 
2.18.0