Re: NSString lowercaseString

[Richard Frith-Macdonald]

On 3 Aug 2012, at 18:21, Sebastian Reitenbach wrote:

> 
> On Thursday, August 2, 2012 11:48 CEST, Richard Frith-Macdonald 
> <richard@tiptree.demon.co.uk> wrote: 
> 
>> 
>> On 1 Aug 2012, at 19:04, Sebastian Reitenbach wrote:
>> 
>>> Hi,
>>> 
>>> On Wednesday, August 1, 2012 18:00 CEST, "Sebastian Reitenbach" 
>>> <sebastia@l00-bugdead-prods.de> wrote: 
>>> 
>>> On the Apple documentation I found -[NSString lowercaseStringWithLocale:] 
>>> bug GNUstep doesn't seem to have that.
>>> When it would be available, would that maybe help me?
>> 
>> I've kept out of this because I don't have an easy answer ... I'm not even 
>> sure what the problem is  (or more likely, problems are).
>> 
>> However, it doesn't appear that anyone else has spotted anything simple 
>> either, and while we don't understand the problem, it seems premature to be 
>> looking for a new method to provide the solution.
>> 
>> So, can I please suggest looking at this slowly and systematically.
>> 
>> It seems possible to me that there are are a variety of areas where a 
>> problem may be, probably four main ones:
>> 
>> 1. what is the compiler putting into the binary?
>> 2. how is base interpreting what it finds?
>> 3. how is the case conversion being done?
>> 4. is there a problem logging it?
>> 
>> The way to find all this out is by stepping through the executable in gdb 
>> and examining what you find there.
>> 
>> If you debug the test program setting a break point immediately before the 
>> literal string is used, you can step into the methods.
>> 
>> eg. for      NSString *l = [@"TöÖst" lowercaseString];
>> you can step into the lowercaseString method and look at the receiver and 
>> see what class it is, and look  at the underlying character data in the 
>> literal string and see whether the compiler has actually generated UTF-8
>> you can then step through and see how characters are being converted to  
>> lowercase etc.
>> Is the lowercaseString implementation from the base NSString class the one 
>> being used?
>> Is the uni_tolower() function working?
>> and so on.
>> 
>> 
> 
> So far I only tried to debug the version on OpenBSD, because there I have the 
> OGo problem. On the Linux box, there I also have an older version installed 
> maybe, 
> and therefore and cannot easily upgrade, to make it better comparable. The 
> thing I just recognized, is on Linux I get:
> 2012-08-01 08:49:57.974 lowercase[16574] autorelease called without pool for 
> object (0x72dce8) of class GSCInlineString in thread <NSThread: 0x6b0cc8>
> and on OpenBSD:
> 2012-08-01 10:38:52.851 lowercase[5483] autorelease called without pool for 
> object (0x209c1c5c8) of class GSUnicodeInlineString in thread <NSThread: 
> 0x20750be08>
> the difference of the GSCInlineString vs. GSUnicodeInlineString

This most probably reflects the version of gnustep you have installed:

On OpenBSD you get a unicode string because base supports utf-8 in string 
literals and has converted the utf-8 literal to a unicode string in order to do 
the case conversion.

On GNU/Linux you get a C string because you are probably using an older version 
of base where only ascii characters were legal in string literals.

So, in the second case (assuming this really is an older version of base beign 
used) the application code is buggy because it contains an illegal string 
literal: the documentation for OpenStep and GNUstep used to say behavior was 
'undefined' if you put non-ascii data in a string literal ... so *any* behavior 
by the base library is correct, and the program needs fixing.

It's the first case we need to look at... using an up to date version of 
GNUstep where support for utf8 in string literals has been added.  I've just 
modified svn trunk to check in more places that the content of a literal really 
looks like valid utf8, so with luck if you update base, you might get an 
exception raised if your toolchain has compiled non-utf8 data.  Please give it 
a try.