[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Creating a GNUstep desktop on Solaris

From: Alexander Malmberg
Subject: Re: Creating a GNUstep desktop on Solaris
Date: Tue, 11 Jan 2005 01:52:19 +0100
User-agent: Mozilla Thunderbird 0.9 (X11/20041124)

Adam Fedor wrote:
Perhaps I should be more specific, much as I love to delve into compiler and language internals :-) Of course the above code will not compile (I said so, in fact). It's really just pseudo-code for a similar problem that occurs in Objective-C due to function pointer indirection. Thus when the compiler sees:

struct foo {int i; int j;};

@interface Foo(Category)
- (struct foo) structReturn;

int main()
struct foo bar = [(Foo *)nil structReturn];

(which is perfectly legal code), it converts the method call into something like (warning: pseudo-like code here):

struct foo bar = objc_msg_send(nil, @selector(structReturn), args);

With the GNU runtime, you get something like:

IMP _temp = objc_msg_lookup(nil, @selector(...));

struct foo bar = ((funky cast)_temp)(nil, @selector(...), args);

which in turn performs some magic to make sure the proper type gets returned for the selector.

That's the funky cast part. :) Normally, the function pointer will be cast to the "right" function pointer type (i.e. with the correct return type and argument types), in the sense that it will match the types that the actual function expects. However, there's only one nil method, so it can't be correct for all possible combinations of types.

Most of the time, with most ABI, this is harmless. However, structure returns are special. In most (it seems) ABI:s, structures are returned by copying them into memory reserved by the caller, so the function really has an extra hidden argument where the caller passes in this address. The nil method doesn't expect to return a structure, so it doesn't know about these hidden arguments or other special handling, and this breaks things.

I don't know exactly how this breaks solaris, but fwiw, it isn't harmless on x86, either. For some reason, the called function is responsible for popping the extra argument off the stack (unlike all other arguments, which are popped by the caller). Thus, the caller will push an extra argument on the stack, but the nil function won't pop it, so you wind up with a stack pointer that's off by 4 bytes after the nil function returns. (And, of course, the memory where the caller expects the structure to be will be completely uninitialized.)

Most of the time, this won't hurt you (stack references usually go through the frame pointer, which is unhurt, and the stack pointer is restored later), but it isn't hard to construct cases where this will cause a crash.

Anyway, the moral of all this is that you should never send messages with structure returns if the receiver might be nil.

(To be extra correct, you should never send messages that don't return object pointers if the receiver might be nil.)

(I believe it would be possible to hack libobjc so that the nil method can detect this and warn even on systems where this won't (always) cause a crash. This should make it easier to find these bugs.)

- Alexander Malmberg

reply via email to

[Prev in Thread] Current Thread [Next in Thread]