Re: GNU gettext 0.20.1 released

[Marius Bakke]

Hi Miguel,

Miguel <address@hidden> writes:

> Hi Tobias,
>
> Tobias Geerinckx-Rice <address@hidden>:
>> Bruno,
>> 
>> Wow.  Thank you for this great summary!  Would that all projects 
>> published such clear (and custom) release notes…  <3
>> 
>> I see that gnu/packages/gettext.scm has a nice chronological list 
>> of copyright lines, which does make it appear as if I'm the 
>> current packager of gettext in Guix.  However, the Guix project 
>> doesn't have this notion of (package) maintainer: everyone 
>> packages, fixes, and updates what they can whenever they can. 
>> This might change in future but it works rather well now.
>> 
>> For that reason, I'm CC'ing the address@hidden list.  I 
>> encourage you to add it to your own for future releases.
>> 
>> I'm having some trouble with the actual upgrade but I'll save that 
>> for a reply.
>
> What are the issues you have?  I could help with that. :-)
>
> I see a big point that need special care: gettext-tools now depends on
> libtextstyle, so gettext-boot0 will definitely fail if only
> gettext-minimal is updated to the next version.
>
> This is going to be a big big update in any case.  We can avoid
> the new bootstrap, keeping 0.19.8.1 for it, but we should update
> gettext-minimal so it may not worth the effort as probably almost all
> the packages will be rebuilt.
>
> My snippet for building it is this one, with some code from
> gettext-minimal and it need some work:
> -------------------->8--------------------
> (define-public gettext-next
>   (package (inherit gettext-minimal)
>     (name "gettext-next")
>     (version "0.20.1")
>     (source (origin
>              (method url-fetch)
>              (uri (string-append "mirror://gnu/gettext/gettext-"
>                                  version ".tar.gz"))
>              (sha256
>               (base32
>                "0p3zwkk27wm2m2ccfqm57nj7vqkmfpn7ja1nf65zmhz8qqs5chb6"))))
>     (inputs
>      `(("xml2" ,libxml2)
>        ;; Avoid dependency cycles
>        ("unistring" ,(@ (gnu packages libunistring) libunistring))
>        ("ncurses" ,(@ (gnu packages ncurses) ncurses))))
>     (arguments
>      `(#:configure-flags
>        (list "--with-included-libunistring=no"
>              "--with-included-libxml=no"
>              (string-append "--with-libxml2-prefix="
>                             (assoc-ref %build-inputs "xml2"))
>              (string-append "--with-libncurses-prefix="
>                             (assoc-ref %build-inputs "ncurses"))
>              (string-append "--with-libtermcap-prefix="
>                             (assoc-ref %build-inputs "ncurses"))
>              (string-append "--with-libunistring-prefix="
>                             (assoc-ref %build-inputs "unistring")))
>        #:phases
>        (modify-phases %standard-phases
>         (add-before 'configure 'patch-fixed-paths
>          (lambda* (#:key inputs #:allow-other-keys)
>            (let* ((bash (which "sh")))
>              (substitute* '("gettext-tools/config.h.in"
>                             "gettext-tools/gnulib-tests/init.sh"
>                             "gettext-tools/tests/init.sh"
>                             "gettext-tools/system-tests/run-test")
>                (("/bin/sh")
>                 bash))
>              (substitute* '("gettext-tools/src/project-id"
>                             "gettext-tools/projects/KDE/trigger"
>                             "gettext-tools/projects/GNOME/trigger")
>                  (("/bin/pwd")
>                   "pwd"))
>              #t)))
>
>         (add-before 'check 'patch-tests
>          (lambda* (#:key inputs #:allow-other-keys)
>            (let* ((bash (which "sh")))
>              ;; Some of the files we're patching are
>              ;; ISO-8859-1-encoded, so choose it as the default
>              ;; encoding so the byte encoding is preserved.
>              (with-fluids ((%default-port-encoding #f))
>                (substitute*
>                    (find-files "gettext-tools/tests"
>                                "^(lang-sh|msg(exec|filter)-[0-9])")
>                  (("#![[:blank:]]/bin/sh")
>                   (format #f "#!~a" bash)))
>
>                #t)))))
>
>        ;; When tests fail, we want to know the details.
>        #:make-flags '("VERBOSE=yes")))))
> --------------------8<--------------------

Thanks for this!  I came up with the following patch to keep the
bootstrap mostly unchanged:

From a53560b0ffed27cf99ae7bc7aa2d1fc19816c5a6 Mon Sep 17 00:00:00 2001
From: Marius Bakke <address@hidden>
Date: Sun, 30 Jun 2019 16:46:37 +0200
Subject: [PATCH] gnu: gettext: Update to 0.20.1.

* gnu/packages/gettext.scm (gettext-minimal): Update to 0.20.1.
[source](snippet): Remove.
[inputs]: Remove EXPAT.  Add LIBUNISTRING, LIBXML2, and NCURSES.
[arguments]: Add #:configure-flags and a "patch-fixed-paths" phase.  Delete
"link-expat" phase.
* gnu/packages/commencement.scm (gettext-boot0): Stay on version 0.19.8.1.
[arguments]: Adjust accordingly.

Co-authored-by: Miguel <address@hidden>
---
 gnu/packages/commencement.scm | 47 ++++++++++++-----------
 gnu/packages/gettext.scm      | 72 +++++++++++++++++++----------------
 2 files changed, 65 insertions(+), 54 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 10b0bd67f5..2dcdb175a8 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2015,30 +2015,33 @@ exec ~a/bin/~a-~a -B~a/lib -Wl,-dynamic-linker 
-Wl,~a/~a \"$@\"~%"
   (let ((gettext-minimal
          (package (inherit gettext-minimal)
            (name "gettext-boot0")
+           ;; GNU Gettext 0.20 and later depends on LibXML and ncurses.  To
+           ;; simplify bootstrapping, we stick to this older version here.
+           (version "0.19.8.1")
+           (source (origin
+                     (method url-fetch)
+                     (uri (string-append "mirror://gnu/gettext/gettext-"
+                                         version ".tar.gz"))
+                     (sha256
+                      (base32
+                       
"0hsw28f9q9xaggjlsdp2qmbp2rbd1mp0njzan2ld9kiqwkq2m57z"))))
            (inputs '())                           ;zero dependencies
            (arguments
-            (substitute-keyword-arguments
-                `(#:tests? #f
-                  ,@(package-arguments gettext-minimal))
-              ((#:phases phases)
-               `(modify-phases ,phases
-                  ;; Build only the tools.
-                  (add-after 'unpack 'chdir
-                             (lambda _
-                               (chdir "gettext-tools")
-                               #t))
-
-                  ;; Some test programs require pthreads, which we don't have.
-                  (add-before 'configure 'no-test-programs
-                              (lambda _
-                                (substitute* "tests/Makefile.in"
-                                  (("^PROGRAMS =.*$")
-                                   "PROGRAMS =\n"))
-                                #t))
-
-                  ;; Don't try to link against libexpat.
-                  (delete 'link-expat)
-                  (delete 'patch-tests))))))))
+            `(#:tests? #f
+              #:phases (modify-phases %standard-phases
+                         ;; Build only the tools.
+                         (add-after 'unpack 'chdir
+                           (lambda _
+                             (chdir "gettext-tools")
+                             #t))
+
+                         ;; Some test programs require pthreads, which we 
don't have.
+                         (add-before 'configure 'no-test-programs
+                           (lambda _
+                             (substitute* "tests/Makefile.in"
+                               (("^PROGRAMS =.*$")
+                                "PROGRAMS =\n"))
+                             #t))))))))
     (package-with-bootstrap-guile
      (package-with-explicit-inputs gettext-minimal
                                    %boot1-inputs
diff --git a/gnu/packages/gettext.scm b/gnu/packages/gettext.scm
index ef80af42ea..17593ee1cc 100644
--- a/gnu/packages/gettext.scm
+++ b/gnu/packages/gettext.scm
@@ -4,10 +4,11 @@
 ;;; Copyright © 2015, 2017 Ricardo Wurmus <address@hidden>
 ;;; Copyright © 2016, 2019 Efraim Flashner <address@hidden>
 ;;; Copyright © 2016 Alex Kost <address@hidden>
-;;; Copyright © 2017 Marius Bakke <address@hidden>
+;;; Copyright © 2017, 2019 Marius Bakke <address@hidden>
 ;;; Copyright © 2017 Mathieu Othacehe <address@hidden>
 ;;; Copyright © 2017 Eric Bavier <address@hidden>
 ;;; Copyright © 2018 Tobias Geerinckx-Rice <address@hidden>
+;;; Copyright © 2019 Miguel <address@hidden>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -33,6 +34,8 @@
   #:use-module (guix build-system perl)
   #:use-module (gnu packages docbook)
   #:use-module (gnu packages emacs)
+  #:use-module (gnu packages libunistring)
+  #:use-module (gnu packages ncurses)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages tex)
   #:use-module (gnu packages xml)
@@ -41,34 +44,47 @@
 (define-public gettext-minimal
   (package
     (name "gettext-minimal")
-    (version "0.19.8.1")
+    (version "0.20.1")
     (source (origin
-             (method url-fetch)
-             (uri (string-append "mirror://gnu/gettext/gettext-"
-                                 version ".tar.gz"))
-             (sha256
-              (base32
-               "0hsw28f9q9xaggjlsdp2qmbp2rbd1mp0njzan2ld9kiqwkq2m57z"))
-             (modules '((guix build utils)))
-             (snippet
-              '(begin
-                ;; The gnulib test-lock test is prone to writer starvation
-                ;; with our glibc@2.25, which prefers readers, so disable it.
-                ;; The gnulib commit b20e8afb0b2 should fix this once
-                ;; incorporated here.
-                 (substitute* "gettext-runtime/tests/Makefile.in"
-                   (("TESTS = test-lock\\$\\(EXEEXT\\)") "TESTS ="))
-                 (substitute* "gettext-tools/gnulib-tests/Makefile.in"
-                  (("test-lock\\$\\(EXEEXT\\) ") ""))
-                 #t))))
+              (method url-fetch)
+              (uri (string-append "mirror://gnu/gettext/gettext-"
+                                  version ".tar.gz"))
+              (sha256
+               (base32
+                "0p3zwkk27wm2m2ccfqm57nj7vqkmfpn7ja1nf65zmhz8qqs5chb6"))))
     (build-system gnu-build-system)
     (outputs '("out"
-               "doc"))                            ;8 MiB of HTML
+               "doc"))                            ;9 MiB of HTML
     (inputs
-     `(("expat" ,expat)))
+     `(("libunistring" ,libunistring)
+       ("libxml2" ,libxml2)
+       ("ncurses" ,ncurses)))
     (arguments
-     `(#:phases
+     `(#:configure-flags
+       (list "--with-included-libunistring=no"
+             "--with-included-libxml=no"
+             (string-append "--with-libxml2-prefix="
+                            (assoc-ref %build-inputs "libxml2"))
+             (string-append "--with-libncurses-prefix="
+                            (assoc-ref %build-inputs "ncurses"))
+             (string-append "--with-libtermcap-prefix="
+                            (assoc-ref %build-inputs "ncurses"))
+             (string-append "--with-libunistring-prefix="
+                            (assoc-ref %build-inputs "libunistring")))
+       #:phases
        (modify-phases %standard-phases
+         (add-before 'patch-source-shebangs 'patch-fixed-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* '("gettext-tools/config.h.in"
+                            "gettext-tools/gnulib-tests/init.sh"
+                            "gettext-tools/tests/init.sh"
+                            "gettext-tools/system-tests/run-test")
+               (("/bin/sh") "sh"))
+             (substitute* '("gettext-tools/src/project-id"
+                            "gettext-tools/projects/KDE/trigger"
+                            "gettext-tools/projects/GNOME/trigger")
+               (("/bin/pwd") "pwd"))
+             #t))
         (add-before 'check 'patch-tests
          (lambda* (#:key inputs #:allow-other-keys)
            (let* ((bash (which "sh")))
@@ -92,15 +108,7 @@
                  (("/bin/pwd")
                   "pwd"))
 
-               #t))))
-        (add-before 'configure 'link-expat
-         (lambda _
-           ;; Gettext defaults to opening expat via dlopen on
-           ;; "Linux".  Change to link directly.
-           (substitute* "gettext-tools/configure"
-             (("LIBEXPAT=\"-ldl\"") "LIBEXPAT=\"-ldl -lexpat\"")
-             (("LTLIBEXPAT=\"-ldl\"") "LTLIBEXPAT=\"-ldl -lexpat\""))
-           #t)))
+               #t)))))
 
        ;; When tests fail, we want to know the details.
        #:make-flags '("VERBOSE=yes")))
-- 
2.22.0

Does it look correct to you?

Do you know if it's possible to delete the bundled copies of libxml and
libunistring entirely?  I got build errors with this source snippet:

--8<---------------cut here---------------start------------->8---
              (modules '((guix build utils)))
              (snippet
               '(begin
                  ;; Delete the bundled copies of libxml2 and libunistring.
                  (for-each delete-file-recursively
                            (find-files "." "^(unistr|libxml)$"
                                        #:directories? #t))
                  #t))
--8<---------------cut here---------------end--------------->8---

Anyway the new gettext does seem to work fine.  It has grown from 78.9
to 97.9 MiB (this is on the 'core-updates' branch):

$ ./pre-inst-env guix size gettext-minimal
store item                                                       total    self
/gnu/store/ycnva13hfplvdc0wp9ckfa909qlhyra6-glibc-2.28              37.5    
36.0  36.7%
/gnu/store/67z1imya4k73rv8iys59lnslxz3k78z4-gcc-7.4.0-lib           70.2    
32.6  33.3%
/gnu/store/0nv03as1mvmw4ii72nirp14857fd4x0w-gettext-minimal-0.20.1    97.9     
9.4   9.6%
/gnu/store/v0gk70nylmzwadwsavvdwxm59i1c77w8-libxml2-2.9.9           80.3     
7.9   8.1%
/gnu/store/094gw83fgil2scrwzbsgw6dxb7zg3g6m-ncurses-6.1-20190609    76.0     
5.9   6.0%
/gnu/store/2xx5pv1zb7v0cw42p25hdgc9yjgpazdn-libunistring-0.9.10     72.5     
2.4   2.4%
/gnu/store/ld746xji0id9gcflm1v645d71y55pnbw-bash-static-5.0.7        1.6     
1.6   1.6%
/gnu/store/argczg2hxrzh5m11dbcz0gghg1cpgzhd-bash-minimal-5.0.7      38.6     
1.0   1.1%
/gnu/store/3pm8plhp2zrybnj59vdzq0r0vjvvnim8-xz-5.2.4                72.1     
0.9   0.9%
/gnu/store/nl92qmy1b61biqzm0cjg2hh9f47qf3w1-zlib-1.2.11             70.4     
0.2   0.2%
total: 97.9 MiB

signature.asc
Description: PGP signature