Question: Why does 'es' fail when 'es_ES' works fine?

[Dan Libby]

Hi, this is not really a bug report.  It is more of a question about 
gettext & gnu/linux locale behavior and why it is not matching my 
expectations.  If this is not the proper forward for such a question, or 
if this is already answered somewhere please point me towards the answer 
or appropriate forum.

My problem is that gettext() is not returning the translated strings 
when a two letter ISO 639 language code is used (via setlocale() or 
'LANG', "LANGUAGE', etc environment variables).  Yet it does work when I 
pass the language code plus country code.

This is a problem for me because web browsers allow users to specify eg: 
'en' or 'es' without specifying a country, and I use the 
HTTP_ACCEPT_LANGUAGE http header to switch among languages.  There is 
also a mechanism on the website for the user to override the browser 
setting, and again that does not ask for a country.  It is far more 
important that the languge be spanish and not english than that it be 
Costa Rican spanish and not Spanish spanish.

The problem seems to be related to my locale setup, and it is system 
wide.  I get the same behavior from various programs (eg wget) if I set 
the 'LANG' environment var to 'es' I still get english, but 'es_ES' 
gives spanish.   I have spent the day looking around the net and have 
not found any clear explanation to why this happening or how to fix it.  
This link contains several posts from people who have run into the same 
problem using PHP gettext.

http://www.php.net/gettext

I have tried adding 'es' to my /usr/share/locales/locale.alias file, 
using the same value as 'spanish'.  However, that does not do anything.  
Further, it is not really a workable solution.  I need to be able to 
deploy this web application on multiple servers and just have it "work", 
not instruct users to muck around with files they may not even have 
access to.  And indeed, it is not just spanish, this is a problem with 
all 4 language codes I have tried so far.

Incidentally, and before someone suggests it, I did manage to make the 
specific case of 'es' work by using the following command:

localedef -c -i es_ES -f ISO-8859-1 es

But I'm not entirely sure why that worked, and anyway it doesn't solve 
the general case or the deployment to multiple machines issue.


So really, it boils down to:

1)  How can I make all the two letter language codes work as 
expected?    ( preferably without changing machine conf, eg is there a 
gettext API or setting I can use in my application? )

2) Out of curiosity, why does gettext care so much about locales?  Maybe 
I want to add a 'jive' or 'swedish chef' translation to my program.  
Does that mean I have to also create a locale definition for 'jive' and 
'swedish chef' on each machine that my app gets installed to?  It would 
seem to make more sense to decouple them so that gettext knows which 
encoding to use for 'jive', and otherwise it is just another message 
catalog...


In case it is of any relevance, I am running an up-to-date Gentoo linux 
with libc-2.3.3.