[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Objective-C and Smalltalk; speed of message send

From: Richard Frith-Macdonald
Subject: Re: Objective-C and Smalltalk; speed of message send
Date: Sun, 8 Aug 2004 08:07:19 +0100

On 8 Aug 2004, at 05:08, Lyndon Tremblay wrote:

"5 timesRepeat: [ Transcript show: 0 tinyBenchmarks; cr ]"

The average of the 3 highest results were chosen (see
http://minnow.cc.gatech.edu/squeak/768) . Ports were done to GNU Smalltalk,
C and Objective-C - GCC 3.4 in all cases)

Squeak 3.8alpha (VM compiled at -O3, populated image),
84563764 bytecodes/sec; 3939081 sends/sec

GNU Smalltalk 2.1.8 (no JIT),
52179161 bytecodes/sec; 3667234 sends/sec

Both compiled at -O3,
C Function - 1938968 calls/sec
Objective-C Message - 824973 sends/sec

This is under a system with stable load, tested multiple times. I've had the Squeak test at "144m bytecodes/sec; 7.7m sends/sec", so perhaps an active image had an effect here. Um, tell me what I have missed, or what is really
going on. I am surprised enough to not believe it is a fair test.

Absolutely no offence intended but ... it's not so much that it's not a fair test,
rather it is a really bad benchmark.

I ran the ObjC benchmark on my system, and found that it reported about
290,000 function calls per second and about 1,600,000 method sends
per second ... since it's obviously ridiculous to say that a method send is three times as fast as a function call, I looked at the code with a bit of care ...

In the first place, the output messages saying calls/sec or sends/sec are not telling the truth, as the benchmark is not measuring calls or sends per second.

In the second place, once you have realised that the the benchmark is not measuring what it says, you then have to realise that it's not really measuring anything meaningful at all. What it is actually doing is finding the smallest fibonacci number that it takes a second or more to calculate, *then* it is adjusting that number in proportion to the time actually taken, presumably
in an attempt to normalise it to the second in some way.
However, since fibonacci numbers are non-linear, the result actually varies in ways which are too horrible to attempt to work out. One extra call will
increase the result by roughly 50%, while depending on the actual time
taken, the final adjustment can leave the result unchanged or halve it,
and these two fluctuations are unlinked.

I changed the code to print out the input to the fib function and the time
actually taken .. which gives you a more reasonable idea of performance.

As you can see, the extra information makes it clear that function calls
are significantly faster than method sends, and the original output
values are bogus.

Function call
Fib 40 in time 1529
292870 calls/sec
Fib 40 in time 1499
298731 calls/sec
Fib 40 in time 1530
292679 calls/sec
Fib 40 in time 1536
291535 calls/sec
Fib 40 in time 1516
295382 calls/sec
Message send
Fib 38 in time 1214
1596309 sends/sec
Fib 38 in time 1216
1593683 sends/sec
Fib 38 in time 1252
1547858 sends/sec
Fib 38 in time 1232
1572986 sends/sec
Fib 38 in time 1225
1581975 sends/sec

PS. Coding the fib generation using iteration rather than recursion, and thus avoiding function/method call overheads made things manyorders of magnitude faster.

If all you want to do is measure call overheads, it would be simpler to just time
a number of loop iterations with one call made each time round the loop.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]