Re: Intelligent ways to search parameter space?

[Stephen C. Upton]

I agree with Rick.  I think the whole "complexity" community has a ways
to go in determining a general set of principles for complex systems, or
even for a subset of complex systems. This also applies to having a
general set of tools in which to explore one's complex system.  In
addition to the references that Rick cites, you could also look at the
design of experiments literature - statisticians have been doing more
work in specifically looking at techniques that are specific to
computational experiments, see, e.g., and the references therein:

Evaluating prediction uncertainty in simulation models. McKay, M.D. and
Morrison, J. D. and Upton, S. C. Computer Physics Communications 117, pp
44-51. 1999.

Improving rule bases for agent-based simulations. Graves, T. and Picard,
R. and Upton, S. C. Los Alamos National Laboratory Report LA-UR-00-2566.
2000. (if anyone would like a copy, please email me)

As somewhat of an aside, I am also finishing up a software framework for
using Natural Algorithms (GA's, simulated annealing, ant algorithms,
cultural algorithms) to "search" over simulation spaces. (this is part
of my long overdue dissertation).  The basic idea is to formulate your
search as an optimization problem and then use the Natural Algorithms as
heuristics to find the "minimum" or the set of minimum.  This is the
idea put forth in Miller's article that Rick references, though he just
mentions any optimization algorithm.  I intend to make the software
framework open source, but haven't had the time to look at using
SourceForge or something similar.  I'd welcome any suggestions here.  

Hope that helps - hope it also doesn't sound like endless plugging ;-)

steve




Rick Riolo wrote:
> 
> Well, i don't know about "intelligent searches",
> but there have been works on "adaptive searches",
> e.g.,
> 
> Exploratory Modeling.  Search Through Spaces of Computational Experiments.
>   Bankes, Steve.
>   Proc. Third. Conf on Evolutionary Programming, pp353-360. 1994.
> 
> Active Nonlinear Tests (ANTs) of Complex Simulation Models.
>   Miller, John.
>   Sante Fe Institute Working Paper 96-03-011.
> 
> Of course your question is also related to
> "sensitivity analysis", and most texts on
> simulation discuss "classical" methods for that.
> 
> More generally, it seems to me your question
> is asking "Is there a theory of complex systems"
> and I'd have to say "No."  There are some rough rules of thumb
> (for instance see axelrod and cohen's recent book
> Harnessing Complexity), but at this point the rules
> are generally at the level of
> 
>   "If you turn the mutation rate too low you might get..."
>   "If you turn the mutation rate too high you might get..."
> 
> There are similar, sometimes more specific rules for
> particular models (eg kaufman's nk models), but
> since there isn't really a good taxonomy of complex systems,
> based on some key features/dimensions/mechanisms of the systems,
> i don't think we can talk very effectively about the
> relationship between "general qualities of complex systems"
> and expected behaviors given certain specific parameters
> in a specific model.
> 
> (but maybe i'm too pessimistic on this dark, very cold
> and snowy morning in michigan!)
> 
> - r
> 
> Rick Riolo                           address@hidden
> Center for Study of Complex Systems (CSCS)
> 4477 Randall Lab
> University of Michigan         Ann Arbor MI 48109-1120
> Phone: 734 763 3323                  Fax: 734 763 9267
> http://www.pscs.umich.edu/PEOPLE/rlr-home.html
> 
> On Thu, 7 Dec 2000, Ken Gosier wrote:
> 
> > Date: Thu, 07 Dec 2000 07:52:29 -0500
> > From: Ken Gosier <address@hidden>
> > Reply-To: address@hidden
> > To: address@hidden
> > Subject: Intelligent ways to search parameter space?
> >
> > So, I've been working a while now on making an extension to the ASM
> > model for the stock market. I've extended it to many assets, and 2 types
> > of agents.
> >
> > I've got it up and running, which is very cool, and I have the
> > LispArchiver hooked in, so it's nice and easy to make different runs for
> > different parameters. I don't yet have a batch mode, but I can figure
> > that out as well.
> >
> > At this point my main task is to start searching parameter space, to
> > find interesting behavior. But: there are ~50 parameters to this model,
> > so exhaustively searching through parameter space in a brute force way
> > is not feasible. Granted, there are only about 5-10 params that I
> > initially care about messing around with, but still it's a daunting
> > task.
> >
> > My question is: are there reference materials out there, which provide
> > some guide to intelligently searching parameter space? I was wondering
> > if there's been any work saying something like: These are general
> > qualities of different types of complex systems. To produce these
> > qualitatively different types of behavior, then you should follow this
> > strategy in setting up the model and/or parameters.
> >
> > I've already managed to change the behavior of the model from completely
> > unstable, to mostly stable, just by using financial knowledge to set the
> > parameters realistically.
> >
> > Apologies if this question is overly vague. I know there's lots of good
> > materials out there. (I know the Michigan Complex Systems page has a
> > good readings page about complex systems.) But, if someone knows some
> > materials in this specific area, it would be quite helpful. Many
> > thanks--
> >
> > Ken Gosier
> > address@hidden
> > address@hidden
> >
> >
> >                   ==================================
> >    Swarm-Modelling is for discussion of Simulation and Modelling techniques
> >    esp. using Swarm.  For list administration needs (esp. [un]subscribing),
> >    please send a message to <address@hidden> with "help" in the
> >    body of the message.
> >                   ==================================
> >
> 
>                   ==================================
>    Swarm-Modelling is for discussion of Simulation and Modelling techniques
>    esp. using Swarm.  For list administration needs (esp. [un]subscribing),
>    please send a message to <address@hidden> with "help" in the
>    body of the message.
>                   ==================================



                  ==================================
   Swarm-Modelling is for discussion of Simulation and Modelling techniques
   esp. using Swarm.  For list administration needs (esp. [un]subscribing),
   please send a message to <address@hidden> with "help" in the
   body of the message.
                  ==================================