Presenting Results (was Re: Sub-swarms)

[Paul Johnson]

This touches on an issue that I'm concerned about. How do you
present simulation results to a general scientific audience?  The
notion that Swarm is a standardizing simulation toolkit was very
appealing to me last  year when I started, but I wish I had more ideas about
how these things can be presented (other than "look at all these cool colored
bugs and the pulsating background...).  In the past, the most useful
thing for me has been to show a series of scatterplots of important endogenous
variables, one for each setting of the parameters.  But that gets out of hand
pretty quickly--too many charts, not enough time.

The other thing I've wondered about is the statistical analysis of results.
I'm often interested in showing that two settings of simulation parameters do
or do not cause a significant change in system behavior. Comparing the two
distributions, one can take advantage of some facts.
In particular, I've considered the analogy to a "matched pairs" analysis.  If
you think of simulation runs under identical circumstances as one distribution,
and you change one parameter and repeat the runs with everything else
identical, including the random number seeds, it
is like having "matched pairs" to compare.  The standard error of the difference
of means is much much smaller with matched pairs and test become much more
powerful.  I wonder how many other little tips like that there are left to be
articulate....
pj

On 26-Feb-98 Benedikt Stefansson wrote:
>      Hi,
> 
> Alex wrote:
> 
>      I am really trying to push
>      the Swarm envelope and re-usability as much as possible.  I
>      believe
>      that a good general purpose experiment manager "library" is
>      vital to
>      Swarm, and definitely needs more attention.
> 
> I would like to second this last statement. I think we should have a
> discussion at Swarmfest (and on-line here for those who are not able to
> attend) about the best methods to
>     a) control experiments
>     b) extract and visualize data from multiple runs.
> 
> I'm sure that there is no one solution that fits all, but it will be
> very useful to share hard won experience and learn more about the types
> of solutions others have come up with.
> 
> Quite a few Swarmers for example have rolled their own Perl scripts or
> used other methods to control parameter sweeps in experiments, which may
> be more efficient than trying to do the same from within Swarm (for
> example it enables poor-man's-parallelism on multiprocessor machines).
> 
> But the ideas that Alex mentioned in his post certainly seem intriguing
> - who wouldn't want to be able to "fast forward" through initial runs
> and then visualize at will when something interesting comes up?
> 
> On point b) - I've not been able to find a satisfying way to treat data
> sets from ABMs, which I would like to manipulate and visualize with an
> object oriented relational data base. I'm sure that the GIS people know
> much more about the types of tools that are out there and look forward
> to learning from them.
> 
> -Benedikt
> 
> ------------------
> Benedikt Stefansson                 address@hidden
> Department of Economics, UCLA       Fax. (310) 825-9528
> Los Angeles, CA 90095-1477          Tel. (310) 825-1777
> 
> 
> 
>                   ==================================
>    Swarm-Support is for discussion of the technical details of the day
>    to day usage of Swarm.  For list administration needs (esp.
>    [un]subscribing), please send a message to <address@hidden>
>    with "help" in the body of the message.
>                   ==================================

----------------------------------
E-Mail: Paul Johnson <address@hidden>
Date: 26-Feb-98
Time: 21:17:17

This message was sent by XFMail
----------------------------------

                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.
                  ==================================