Re: Unicode and GNUstep (ö --> ?)

[Pascal Bourguignon]

> From: Andreas Hoeschler <ahoesch@smartsoft.de>
> Date: Fri, 10 May 2002 15:54:38 +0200
> 
> Hi all,
> 
> When I do NSLog(@"Andreas Höschler") on MacOSX I get
> 
>       2002-05-10 15:49:02.252 FBTest[12868] Andreas H\\366schler
> 
> When I do the same on GNUstep I get
> 
>       May 10 15:45:50 FBTest[2073] Andreas H?schler

The  behavior of stringWithCString  and cString  is determined  by the
default  encoding  that  can  be  set with  the  environment  variable
GNUSTEP_STRING_ENCODING.    If   it's   not   set,  it   defaults   to
NSISOLatin1StringEncoding. 

Do you get this  '?' on a xterm or in a  file?  Does you xterm support
the ISO-8859-1 encoding?


> It' s not that I am wondering how to log my name but the same occurs
> when generating SQL. My program works perfectly on MacOSX (lastname =
> "Höschler" --> returning one row) but on GNUstep I get SQL like
> 
>       SELECT t0."PERSONID",t0."FIRSTNAME",t0."LASTNAME" FROM "PERSON" as
> t0 WHERE t0."LASTNAME" = 'H?schler';

Where to you probe this SQL from?  How do you get it?

 
> and of course no rows back. What is the problem here? Why is "ö"
> converted to "?" and by whom?

Yes, by whom?  Quite hairy!

There is some  support for various encodings only at  the level of the
specific  adaptors, in  the sense  that with  the login  info  one may
specify  the encoding  the  data base  should  use to  speak with  the
application (NLS_LANG in the case of Oracle7Adaptor). EOF and gdl work
generally  with  NSString,  therefore  in UNICODE  (automatically,  if
needed).





So, normally,  you have an EOQualifier  somewhere to which  you give a
NSString containing @"Andreas Höschler".

When the  qualifier is needed to  build the where clause,  it's sent a
expressionValueForContext:   message,  from   where   it  forwards   a
formatValue:forAttribute: messsage to the adaptor.

So the question is, what adaptor do you use?




Supposing   you're   using   the   Oracle7   (which   I   know),   the
formatValue:forAttribute: has three cases:

    -(id)formatValue:(id)value forAttribute:(EOAttribute*)attribute
    {
    // SEE: This implementation is insufficient. Let's have a look at Oracle 
Doc.
        if([self oracleTypeIsStringType:[attribute externalType]]){
            return([self quotedStringValue:value]);
        }else if([value isKindOfClass:[NSDate class]]){
            return([self dateValue:value]);
        }else{
            return([super formatValue:value forAttribute:attribute]);
        }
    }//formatValue:;

Normally, for a string attribute, the attribute externalType should be
string so  the quotedStringValue  of Oracle7Adaptor would  be invoked.
This method  works with  unichars, so if  the value string  contains a
"ö", it should be passed along to the where clause of the select.

In last  resort, a  formatValue:forAttribute: message is  forwarded to
the super class, EOAdaptor, where  a similar processing should be done
anyway.

(for a NSString value argument, -[EOAdaptor formatValue:forAttribute:]
calls  -[NSString  expressionValueForContext:]  which  is  defined  in
EOExpressionArray.m as returning self).


So,  if  I'm  not  wrong  in  my reading  of  the  sources,  with  the
Oracle7Adaptor, I would expect a correct select clause.


However, from then on, things are more delicate.


The Oracle7Channel evaluateExpression: send a parseSQL: message to the
private  clsas  O2CI7Cursor  which  converts  the  expression  into  a
cString. Oops.


    -(BOOL)parseSQL:(NSString*)sqlStatement
    {
        [self cleanErrorMessages];      
        if(isOpen){
            BOOL result;
            text* sqlText=(text*)malloc(sizeof(text)
                                        *([sqlStatement cStringLength]+1));
            [sqlStatement getCString:sqlText];
            // We're not sure that oparse don't change sqlText;
            // it's not (const text*).
            //NSLog(@"oparse(%@)\n",sqlStatement);
            if(oparse(&cda,sqlText,-1,0,1)){
                [self reportErrorFromOCI];
                result=NO;
            }else{
                result=YES;
            }
            free(sqlText);
            return(result);
        }else{
            [self reportErrorNotOpen];
            return(NO);
        }
    }//parseSQL:;

This  method should  rather explictely  determine the  encoding that's
been configured for  the channel between the application  and the data
base, and try to convert not to  a C string, but to a string encodeded
with this encoding.


In the  other direction, it's not  better, since data  coming from the
data base is processed in Oracle7Buffer.m, and converted to a NSString
from a C string:

    -(NSString*)asString
    {
        if([self isNullOrWhat]){
            return([EONull null]);
        }else{
            return([NSString stringWithCString:buffer]);
        }
    }//asString;


Here too, this method (and the other similar methods of Oracle7Buffer)
should rather explictely expect the  data in the encoding specified at
login  time,   and  translate  it   to  UNICODE  and  passed   to  the
stringWithCharacters:length: method instead of stringWithCString:.



However, in  both cases, and for  the encodings that  are supported as
default encodings for C strings, one can expect stringFromCString: and
cString to do the right thing.





One  final note:  of course,  the  encoding you  configure as  default
encoding to be used by  the Oracle7 adaptor with its interchanges with
the database,  should match the encoding configured  in this database,
and that specified in the NLS_LANG login parameter.

-- 
__Pascal_Bourguignon__             (o_ Software patents are endangering
() ASCII ribbon against html email //\ the computer industry all around
/\ and Microsoft attachments.      V_/ the world http://lpf.ai.mit.edu/
1962:DO5K=1.3   2001:my($f)=`fortune`;   http://petition.eurolinux.org/

You're a web designer?   Please read http://www.useit.com/alertbox/ !

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/IT d? s++:++(+++)>++ a C+++  UB+++L++++$S+X++++>$ P- L+++ E++ W++
N++ o-- K- w------ O- M++$ V PS+E++ Y++ PGP++ t+ 5? X+ R !tv b++(+)
DI+++ D++ G++ e+++ h+(++) r? y---? UF++++
------END GEEK CODE BLOCK------