Re: [gnugo-devel] Go as SPEC benchmark?

[Jim Koehler]

Hello Arend,

   Thanks for your thoughts.  Sorry to be slow responding.  My response
to your two specific points is given below.

  I haven't detected a lot of interest in adapting Go for cpu2004, but I
thought I might take a stab at getting it off the ground.  I built it
for an Itanium-based HU-UX system, but the executable aborts right
away.  I want to try another platform or two and investigate some more. 
When I can clearly state my questions, I'll write again.
 
 (more below, in response to points 1 and 2 )

Thanks again.  Regards,  - Jim

Arend Bayer wrote:
> 
> Dear James Koehler,
> 
> personally I, too, would find it very nice to have a Go program or
> specifically GNU Go in the spec benchmark suite. Without knowing much
> about professional benchmark requirements, I'd also think that it is
> well suited as a benchmark, as the performance critical code is spread
> among many very different functions.
> Also, we do aim at complete reproducability among different platforms,
> so additional testing with respect to this in the course of getting
> it accepted for SPEC would be very welcome.
> 
> I see some specific problems, however, maybe you can comment on these:
> 
> 1. About 5% of the time is spent doing floating point computations
> (whereas the rest of the code is 98% integer computations). Is it a
> pre-condition for SPEC acceptance to depend either completely on integer
> or completely on floating point performance?
> 
> (It is not much code and could be re-written using fixed-point integer
> arithmetic.)

   This may be a minor issue.  It would be better to eliminate the fp
computaitons, but I'm not sure how big a deal 5% is.

> 
> 2. The cache is badly organized for 64bit architectures (i.e. they can
> store less entries if using the same amount of memory). I don't know
> what kind of fairness between 32 and 64 bit architectures is expected
> for SPEC.
> 
> Definitely, one would have to make sure that this problem doesn't affect
> behaviour (which it could at the moment), as this would add serious
> noise to the benchmark results.
> 
   The 32/64 bit difference would actually work the other way.  The
allocation should be done in terms of number of program entities, rather
than in MBytes.  So the 64 bit systems would be on an equal footing as
far as caching goes, but they would have to allocate more memory.  That
is to say, if the 32 bit version running the reference workload can and
does allocate 100K boards, 200K dragons, and 300K whachamacallits, the
64 bit version needs to allocate the same number of things, even if it
takes almost twice the memory in MBytes.

   Currently the program does the allocation based on a requested size
in MBytes.  That is something that would have to be changed.  That might
not be so easy to do.  It might have to be workload dependent, and it
could be kind of messy. 
 

> I am also curious what others in our team think. I'd see sufficient
> overlap with our general goals to make this worth some reasonable
> effort.
> 
> Regards,
> Arend
> 
> /---------------------------------------------------------------------\
> |                             Arend Bayer                             |
> |                          Flodelingsweg 27a                          |
> |                             53121 Bonn                              |
> |                            0228-9813803                             |
> \---------------------------------------------------------------------/

-- 
------------------------------------------------------
James F. Koehler (Jim)         Hewlett-Packard Company
address@hidden             (972) 497-4932
------------------------------------------------------