Re: Frequent locales problems for new users

[Ludovic Courtès]

Hi Leo,

Leo Famulari <address@hidden> skribis:

> On Wed, Mar 18, 2020 at 04:07:22PM +0100, Ludovic Courtès wrote:
>> As for ‘glibc-utf8-locales’ vs. ‘glibc-locales’: the reason for choosing
>> the former by default over the latter is size (14 MiB vs. 917 MiB).
>
> Oof! I was going by the manual, which says 110 MiB. That does change
> things...

Yes, I was also surprised.

The patch below produces a package that includes all the UTF-8 locales
(actually I had written that patch long ago, it feels like we’re running
in circles :-)).

It takes ages to build, and when it’s finally done:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -e '((@@ (gnu packages base) 
make-glibc-utf8-locales/full))' 
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substituting /gnu/store/jdfs3xvlnj272475yja6bjrprfsgnkdd-glibc-2.29...
downloading from 
https://ci.guix.gnu.org/nar/lzip/jdfs3xvlnj272475yja6bjrprfsgnkdd-glibc-2.29...
 glibc-2.29  8.2MiB                                                       
1.8MiB/s 00:05 [##################] 100.0%

building 
/gnu/store/w08zi9vnkd7bxpfvm5lgjyb30i7k7sw4-glibc-supported-utf8-locales.scm.drv...
successfully built 
/gnu/store/w08zi9vnkd7bxpfvm5lgjyb30i7k7sw4-glibc-supported-utf8-locales.scm.drv
building 
/gnu/store/ps6wh05pwjp5b0l9rh2yglv3sggpgcw4-glibc-utf8-locales-2.29.drv...
successfully built 
/gnu/store/ps6wh05pwjp5b0l9rh2yglv3sggpgcw4-glibc-utf8-locales-2.29.drv
/gnu/store/p0knl9ggxk91x87ww702g2x78jxy1vgf-glibc-utf8-locales-2.29
ludo@ribbon ~/src/guix$ guix size 
/gnu/store/p0knl9ggxk91x87ww702g2x78jxy1vgf-glibc-utf8-locales-2.29 | tail -1
total: 855.7 MiB
--8<---------------cut here---------------end--------------->8---

So I think that’s when we reached the conclusion that we needed
parameterized packages to allow users to choose the locale(s) they need
or special support in ‘guix package’.

:-/

Attached is the list of supported UTF-8 locales, 312 in total.

Thoughts?  How do other distros deal with this?  Are we missing some
trick to compress locale data?

Ludo’.

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index e8150708c0..98b413da13 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès 
<address@hidden>
+;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic 
Courtès <address@hidden>
 ;;; Copyright © 2014, 2019 Andreas Enge <address@hidden>
 ;;; Copyright © 2012 Nikita Karetnikov <address@hidden>
 ;;; Copyright © 2014, 2015, 2016, 2018 Mark H Weaver <address@hidden>
@@ -52,6 +52,8 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages gettext)
   #:use-module (guix utils)
+  #:use-module (guix gexp)
+  #:use-module (guix modules)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
@@ -61,6 +63,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
+            %default-utf8-locales
+            make-glibc-utf8-locales
             libiconv-if-needed))
 
 ;;; Commentary:
@@ -1076,7 +1080,12 @@ to the @code{share/locale} sub-directory of this 
package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+  '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
+(define* (make-glibc-utf8-locales glibc #:optional
+                                  (locales %default-utf8-locales)
+                                  (locale-file #f))
   (package
     (name "glibc-utf8-locales")
     (version (package-version glibc))
@@ -1115,10 +1124,17 @@ to the @code{share/locale} sub-directory of this 
package.")
 
                                ;; These are the locales commonly used for
                                ;; tests---e.g., in Guile's i18n tests.
-                               '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+                               ,(if locale-file
+                                    `(call-with-input-file
+                                         (assoc-ref %build-inputs 
"locale-file")
+                                       read)
+                                    `',locales))
                      #t))))
     (native-inputs `(("glibc" ,glibc)
-                     ("gzip" ,gzip)))
+                     ("gzip" ,gzip)
+                     ,@(if locale-file
+                           `(("locale-file" ,locale-file))
+                           '())))
     (synopsis "Small sample of UTF-8 locales")
     (description
      "This package provides a small sample of UTF-8 locales mostly useful in
@@ -1145,6 +1161,40 @@ test environments.")
 (define-public glibc-locales-2.27
   (deprecated-package "glibc-locales-2.27" glibc-locales-2.28))
 
+(define (glibc-supported-locales libc)
+  ((module-ref (resolve-interface '(gnu system locale)) ;FIXME: hack
+               'glibc-supported-locales)
+   libc))
+
+(define* (make-glibc-utf8-locales/full #:optional (glibc glibc))
+  (define utf8-locales
+    (computed-file "glibc-supported-utf8-locales.scm"
+                   #~(begin
+                       (use-modules (srfi srfi-1)
+                                    (ice-9 match)
+                                    (ice-9 pretty-print))
+
+                       (define locales
+                         (call-with-input-file
+                             #+(glibc-supported-locales glibc)
+                           read))
+
+                       (define utf8-locales
+                         (filter-map (match-lambda
+                                       ((name . "UTF-8")
+                                        (if (string-suffix? ".UTF-8" name)
+                                            (string-drop-right name 6)
+                                            name))
+                                       (_ #f))
+                                     locales))
+
+                       (call-with-output-file #$output
+                         (lambda (port)
+                           (pretty-print utf8-locales port))))))
+
+  (make-glibc-utf8-locales glibc #:locale-file utf8-locales))
+
+
 (define-public which
   (package
     (name "which")

txtz2cWMch86x.txt
Description: Text document