Recursively-modular neural network construction in Swarm

[Alex Lancaster]

Dear Hive,

I am writing with a series of question to all those neural network (and
genetic algorithm) modellers out there...

I am attempting to develop a Swarm program which uses a recursively modular
neural network, which is constructed from a genome (the NN is the "brain"
of a "bug" creature with vision which moves about in an artificial
"world"). Recursively modular, in this context, simply means that the
neural architecture has different 'levels', or 'scales' with components at
a given scale having similar structures, giving the entire network a high
degree of self-similarity (dense local connections and sparse global
connections). 

The proposed scheme would work something like the following:

To create a scale "zero" neural network:

1. Start with a "prototype" neurone, with a certain set of empirical
real-world parameters: eg. efficiency, time constant, conductivity, whether
excitatory or inhibitory etc. as specified by the genome.
2. Some part of the genome specifies the number of "copies" of this
prototypical neurone to be made, and applies small perturbations to all of
the parameters of the copies (also specified in the genome).
3. Once these "copies" are made, a set of rules (also specified by the
genome) are invoked to connect these neurones together. Such as:
  A. Join all excitatory neurones together with a conductivity greater than
some      threshold value
  B. Join inhibitory neurones with blah, blah...
4. This would complete the construction of a "scale zero" level.

Simple so far. The next step (the difficult bit!) is to generalise the
above process to apply to construction of any scale network (i.e. scale "n"):

1. Start with a scale "n" network.
2. Make "copies"
3. Perturb the parameters of the copies slightly
4. Use the genome rules to "wire" up all the neurones to create a new scale
"n+1" level network (note that we don't just connect 'scale n' to other
'scale n' networks, this connection process must be general enough to reach
down and connect up the individual neurones, ultimately).

The difficult conceptual bits are:

1. Creating a genetic code that can has a data structure which can be
interpreted unambiguously under crossover and mutation, and simultaneously
allow for it's own evolution (i.e. a parents could have sections of their
code with differing lengths, and still produce legitimate offspring).
2. How to create a recursive procedure and not get caught in a "level-mix-up".

Suggestions on these matters are welcome.

As far as implementing this in Swarm - I'm comfortable enough with the
mechanics of developing a Swarm simulation (I have developed a very simple
model of the above "bugworld" - without the neural network part - in
Swarm). It seems to me that the idea of the recursively modular part of the
neural network, ought to lend itself very well to the Swarm architecture
(i.e. modelling Swarms of Swarms - where a Swarm in this case might be a
"scale n" network for example), but I am having great difficulty converting
this high-level "isomorphism" into a concrete design and implementation. 

I keeping coming back to the fact that the network must be constructed (at
initialisation and during crossover) and then must also work in some sort
of data structure, when working as the "brain" for the bug. I don't want to
have to maintain two separate structures, but the OO-paradigm of
constructors, member functions and messages doesn't seem to be working for
me in this context. Maybe I am just haven't got the "right" way of seeing
the problem, yet. These are tricky issues indeed, I'm sure (and I hope)
that someone else out there in the Swarm community has grappled with these
kind of neural networks.

I seem to recall a discussion of a "brain protocol" earlier in these posts,
does anyone know whether it may have relevance to my problem?

Any takers?

Regards,

Alex Lancaster
////////////////////////////////////////////////////

  Alex Lancaster
  e-mail: <address@hidden>
  tel: +61-2-9565-1990 (H)

////////////////////////////////////////////////////


                  ==================================
   Swarm-Modelling is for discussion of Simulation and Modelling techniques
   esp. using Swarm.  For list administration needs (esp. [un]subscribing),
   please send a message to <address@hidden> with "help" in the
   body of the message.
                  ==================================