Re: [Swarm-Modelling] SWARM on Clusters

[Russell Standish]

On Mon, Feb 02, 2004 at 11:44:30AM +0000, James Marshall wrote:
> Hi,
>   don't forget the easiest form of parallelism... job farming, is easily
> achievable with Swarm. I had this set up on my last project (using Sun
> Grid Engine on a cluster) and it's very handy if you have to do lots of
> sensitivity or robustness analysis. There used to be something called
> Drone to do a similar thing but I'm not sure it's maintained?
>   Also I don't think Russel Standish has contributed to the discussion
> yet but he has lots of experience in this area... check out the High 
> Performance
> Computing Support Unit at the University of New South Wales.
>   Regards,
>     James
> 

Sorry for the deathly silence - been a bit busy dealing with our own
funding agencies :)

We have done some some substantial size simulations of around a
million agents distributed over 100 nodes of a cluster. Scaling was
pretty impressive, with 40 times speedup achieved.

However, this is not with Swarm, but with Ecolab, a "Swarm-inspired"
ABM environment based on C++. The key technology that we use is
something called Graphcode, which is available as separate product,
and also integrated in EcoLab. (Currently beta, as EcoLab.4.3.x, but
full regression tested versions promised real soon now :).

The idea behind Graphcode is "agents on a graph". Graphcode takes care
of migrating agents between processors as necessary, including for
load balancing reasons, and performing the work on a local list of
nodes. Nearest neighbours living on distant nodes are cached prior to
running the computation part.

This is a very flexible paradigm, as Cartesian grid (aka spaces) is a
simple type of graph. However, the topology can be arbitrary, so
potentially useful for things like social networks. The great thing is
that the model scales to very large systems.

Nothing much is written up about Graphcode yet - I hope I can make it
to a SwarmFest to talk about it - if not Michigan, then maybe Italy :)


Now lets say you wanted to use Graphcode with Swarm. One of the things
we did figure out how to do was to embed C++ models into Swarm. (See
Richard Leow's paper from last SwarmFest). It should also be possible
to call back Objective C methods from C++ so as to execute Objective C
agents. We had hoped to knock up a demo where Heatbugs were running in
parallel in Graphcode within the Swarm environment, however funding
for that line of work has basically dried up, and we have other
priorities now :(. It would still be a kewl thing to do if anyone is
interested.

                                Cheers

----------------------------------------------------------------------------
A/Prof Russell Standish                  Director
High Performance Computing Support Unit, Phone 9385 6967, 8308 3119 (mobile)
UNSW SYDNEY 2052                         Fax   9385 6965, 0425 253119 (")
Australia                                address@hidden             
Room 2075, Red Centre                    http://parallel.hpc.unsw.edu.au/rks
            International prefix  +612, Interstate prefix 02
----------------------------------------------------------------------------

pgpsCbwRnucx8.pgp
Description: PGP signature