i18n, gettext support

[Bruno Haible]

Hello,

Jan Nieuwenhuizen asked me (the GNU gettext maintainer) for how to support
Guile programs in gettext.

The idea would be to make libintl bindings at the C level, and add support
for Scheme to xgettext and msgfmt. Common Lisp support has already been done
long ago; the guile support would therefore be a simple clone of it.

I plan to use this sample "hello world" program as an example. Is this OK
with you, the guile developers?

=============================== hello.scm ==================================
address@hidden@ -s
!#
;;; Example for use of GNU gettext.
;;; Copyright (C) 2004 Free Software Foundation, Inc.
;;; This file is in the public domain.

;;; Source code of the GNU guile program.

(use-modules (i18n))
(use-modules (ice-9 format))

(set! (i18n:textdomain) "hello-guile")
(set! (i18n:textdomaindir "hello-guile") "@localedir@/")
(define _ i18n:gettext)

(display (_ "Hello, world!"))
(newline)
(format #t (_ "This program is running as process number ~D.") (getpid))
(newline)
=============================================================================

Attached you find my proposal for the libintl binding module. It is based
on the existing interface in GNU clisp, and adapted for guile. Please
comment on it. You can use this proposal as a base for the documentation
of this interface.

Technically, this binding could be distributed with guile or with gettext.
I would much prefer if it were part of guile, because this frees the
guile user from checking whether the guile-libintl interface is installed
or not.

The ice-9 format string facility with its address@hidden facility is sufficient 
for
i18n. Is it true that most guile programs use format, not printf, for
formatted string output?

> without guile properly understanding multi-byte strings it's a bit limited

Without proper multibyte or Unicode string support, programs can not
do surgery (truncation, uppercase conversion etc.) on strings returned
from i18n:gettext, but it can output them.

Bruno

Internationalization

"Internationalization" means to prepare a program so that it can use multiple national languages and national cultural conventions without requiring further source code changes. Localization means providing the data - mostly textual translations - necessary for an internationalized program to work in a particular language and with particular cultural conventions.

guile supports internationalized Lisp programs, through GNU gettext.

The GNU gettext binding

1. Domain

2. Category

3. Example

GNU gettext is a set of functions, included in guile or the C library, which permit looking up translations of strings through message catalogs. It is also a set of tools which makes the translation maintenance easy for the translator and the program maintainer.

The GNU gettext functions are available in guile in the “i18n” module.

(i18n:gettext MSGID [DOMAIN [CATEGORY]])

returns the translation of the message MSGID, in the given DOMAIN, depending on the given CATEGORY. MSGID should be an ASCII string, and is normally the English message.

(i18n:ngettext MSGID msgid_plural n [DOMAIN [CATEGORY]])

returns the plural form of the translation for of MSGID and n in the given DOMAIN, depending on the given CATEGORY. MSGID and msgid_plural should be ASCII strings, and are normally the English singular and English plural variant of the message, respectively.

1. Domain

The DOMAIN is a string identifier denoting the program that is requesting the translation. The pathname of the message catalog depends on the DOMAIN: usually it is located at TEXTDOMAINDIR/l/LC_MESSAGES/domain.mo, where l is the ISO 639 code of the language. The notion of DOMAIN allows several Lisp programs running in the same image to request translations independently of each other.

Function i18n:textdomain. (i18n:textdomain) is a place that returns the default DOMAIN, used when no DOMAIN argument is passed to the i18n:gettext and i18n:ngettext functions. It is SET!able. SET! i18n:textdomain is usually used during the startup phase of a program. Note that the default DOMAIN is not saved in a memory image. The use of SET! i18n:textdomain is recommended only for programs that are so simple that they will never need more than one DOMAIN.

Function i18n:textdomaindir. (i18n:textdomaindir DOMAIN) is a place that returns the base directory, called TEXTDOMAINDIR above, where the message catalogs for the given DOMAIN are assumed to be installed. It is SET!able. SET! i18n:textdomaindir is usually used during the startup phase of a program, and should be used because only the program knows where its message catalogs are installed. Note that the TEXTDOMAINDIRs are not saved in a memory image.

Function i18n:textdomain-codeset. (i18n:textdomain-codeset DOMAIN) is a place that returns the specified encoding for the strings returned by i18n:gettext and i18n:ngettext. The default value is #f, denoting the default locale encoding. It is SET!able. The value it is set to must be an encoding name known to the system iconv() function. SET! i18n:textdomain-codeset is usually used during the startup phase of a program whose internal string representation is not in the default locale encoding. Note that the TEXTDOMAIN-CODESETs are not saved in a memory image.

2. Category

The CATEGORY argument of the i18n:gettext and i18n:ngettext functions denotes which locale facet the result should depend on. The default value is :LC_MESSAGES. Other possible values are :LC_CTYPE, :LC_TIME, :LC_COLLATE, :LC_MONETARY. The use of these values is useful for users who have a character/time/collation/money handling set differently from the usual message handling. Note that when a CATEGORY argument is used, the message catalog location depends on the CATEGORY: it will be expected at TEXTDOMAINDIR/ll/category/domain.mo.

3. Example

A non-internationalized program simulating a restaurant dialogue might look as follows.

Example 1. prog.scm

(define n (parse-integer (first *ARGS*)))

(format t "~A~%" "'Your command, please?', asked the waiter.")

(format t "address@hidden"
          (if (= n 1) "a piece of cake" "~D pieces of cake")
          n)

After being internationalized, all strings are wrapped in i18n:gettext calls, and i18n:ngettext is used for plurals. Also, i18n:textdomaindir is assigned a value; in our case, for simplicity, the current directory.

Example 2. prog.scm

(set! (textdomain) "prog")
(set! (textdomaindir "prog") "./")

(define n (parse-integer (first *ARGS*)))

(format t "~A~%"
          (gettext "'Your command, please?', asked the waiter."))

(format t "address@hidden"
          (ngettext "a piece of cake" "~D pieces of cake" n)
          n)

For ease of reading, it is customary to define an abbreviation for the i18n:gettext function. An underscore is customary.

Example 3. prog.scm

(set! (textdomaindir "prog") "./")
(define (_ msgid) (gettext msgid "prog"))

(define n (parse-integer (first *ARGS*)))

(format t "~A~%"
          (_"'Your command, please?', asked the waiter."))

(format t "address@hidden"
          (ngettext "a piece of cake" "~D pieces of cake" n "prog")
          n)

Now the program's maintainer creates a message catalog template through the command

bash$ xgettext -o prog.pot prog.scm

Note

xgettext version 0.14.2 or higher is required here.

The message catalog template looks roughly like this.

Example 29.4. prog.pot

msgid "'Your command, please?', asked the waiter."
msgstr ""

msgid "a piece of cake"
msgid_plural "%d pieces of cake"
msgstr[0] ""
msgstr[1] ""

Then a French translator creates a French message catalog

Example 29.5. prog.fr.po

msgid ""
msgstr ""
"Content-Type: text/plain; charset=ISO-8859-1\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"

msgid "'Your command, please?', asked the waiter."
msgstr "«Votre commande, s'il vous plait», dit le garçon."

# Les gateaux allemands sont les meilleurs du monde.
msgid "a piece of cake"
msgid_plural "%d pieces of cake"
msgstr[0] "un morceau de gateau"
msgstr[1] "%d morceaux de gateau"

and sends it to the program's maintainer.

The program's maintainer compiles the catalog as follows:

bash$ mkdir -p ./fr/LC_MESSAGES
bash$ msgfmt -o ./fr/LC_MESSAGES/prog.mo prog.fr.po

When a user in a french locale then runs the program

bash$ guile prog.scm 2

she will get the output

    «Votre commande, s'il vous plait», dit le garçon.
    2 morceaux de gateau