CAP6938 Neuroevolution and Artificial Embryogeny Leaky Integrator Neurons and CTRNNs

1. CAP6938Neuroevolution and Artificial EmbryogenyLeaky Integrator Neurons and CTRNNs Dr. Kenneth Stanley March 6, 2006

2. Artificial Neurons are a Model • Standard activation model • But a real neuron doesn’t have an activation level • Real neurons fire in spike trains • Spikes/second is a rate • Therefore, standard activation can be thought of as outputting a firing rate at discrete timesteps (i.e. rate encoding) Wolfgang Maass, http://www.tu-graz.ac.at/igi/maass

3. What is Lost in Rate Encoding? <> • Timing information • Synchronization • Activity between discrete timesteps 30 Neurons Firing in a monkey’s striate cortex From Krüger and Aiple [Krüger and Aiple, 1988]. Reprinted from www.igi.tugraz.at/ maass/123/node2.html

4. Spikes Can Be Encoded Explicitly • Leaky integrate and fire neurons • Encode each individual spike • Time is represented exactly • Each spike has an associated time • The timing of recent incoming spikes determines whether a neuron will fire • Computationally expensive • Can we do almost as well without encoding every single spike?

5. Yes: Leaky Integrator Neurons (CTRNNS; Continuous Time Recurrent Neural Networks) • Idea: Calculate activation at discrete steps but describe rate of change on a continuous scale • Instead of activating only based on input, include a temporal component of activation that controls the rate at which activation goes up or down • Then the neuron can react to changes in a temporal manner, like spikes

6. Activation Rate Builds and Decays Input to neuron • Incoming activation causes the output level to climb over time • We can sample the rate at any discrete granularity desired • A view is created of temporal dynamics without full spike-event simulation Activation Level (i.e. spike rate) Output over time time

7. What is Leaking In a Leaky Integrator? • The neuron loses potential at a defined rate • Each neuron leaks at its own constant rate • Each neuron integrates at the same constant rate as well Leaking activation level (membrane potential) Activation Level (i.e. spike rate) time

8. Leaky Integrator Equations Leak • Expressing rate of change of activation level: • Apply Euler Integration to derive discrete-time equivalent • Expressing current activation in terms of activation on previous discrete timestep: Real time Between steps Equations from: Blynel, J., and Floreano, D. (2002). Levels of dynamics neural controllers. In Proceedings of the Seventh International Behavior on From Animals to Animats, 272–281.

9. What Can a CTRNN Do? • With the right time constants for each neuron, complex temporal patterns can be generated • That is, the time constants are a new parameter (inside nodes) that can evolve • More powerful than a regular RNN • Capable of generating complex tenporal patterns with no input and no clock

10. Pattern Generation for What? • Walking gaits with no input! Evolution of central pattern generators for bipedal walking in a real-time physics environmentT Reil, P Husbands - Evolutionary Computation, IEEE Transactions on, 2002

11. Reil and Husbands Went on to Found the Company NaturalMotion

12. Pattern Generation for What? • Salamander walking gait • Wing flapping Ijspeert A.J.: A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander, Biological Cybernetics, Vol. 84:5, 2001, pp 331-348. Evolution of neuro-controllers for flapping-wing animats - group of 2 »JB Mouret, S Doncieux, L Muratet, T Druot, JA … - Proceedings of the Journees MicroDrones, Toulouse, 2004

13. Maybe Good for Other Things with Temporal Patterning • Music? • Picture Drawing? (certain types of patterns) • Tasks that we typically do not conceive in terms of patterns? • Learning tasks (better than a simple RNN?; Blynel and Floreano 2002 paper) • Largely unexplored • How far away from the benefits of a true spiking model?

14. Leaky NEAT • There is a rough, largely untested leakyNEAT at the NEAT Users Group files section: • http://groups.yahoo.com/group/neat/files/ • Introduces a new activation function and new time constant parameter in the nodes • The topology of most CTRNNs in the past was determined completely by the researcher

15. Next Topic: Non-neural NEAT, Closing Remarks on Survey Portion of Class • Complexification and protection of innovation in non-neural structures • Example: Cellular Automata neighborhood functions • What have we learned, what is its significance, and where does the field stand? Read for 3/8/06: Mitchell Textbook pp. 44-55 (Evolving Cellular Automata) think about: How would NEAT apply to this task? Homework due 3/8/06 (see next slide)

16. Homework Due 3/8/06 • Genetic operators all working: • Mating two genomes: mate_multipoint, mate_multipoint_avg, others • Compatibility measuring: return distance of two genomes from each other based on coefficients in compatibility equation and historical markings • Structural mutations: mutate_add_link, mutate_add_node, others • Weight/parameter mutations: mutate_link_weights, mutating other parameters • Special mutations: mutate_link_enable_toggle (toggle enable flag), etc. • Special restrictions: control probability of certain types of mutations such as adding a recurrent connection vs. a feedforward connection • Turn in summary, code, and examples demonstrating that all functions work. Must include checks that phenotypes from genotypes that are new or altered are created properly and work.

17. Project Milestones (25% of grade) • 2/6: Initial proposal and project description • 2/15: Domain and phenotype code and examples • 2/27: Genes and Genotype to Phenotype mapping • 3/8: Genetic operators all working • 3/27: Population level and main loop working • 4/10: Final project and presentation due (75% of grade)