CAP6938 Neuroevolution and Developmental Encoding Leaky Integrator Neurons and CTRNNs

1. CAP6938Neuroevolution and Developmental EncodingLeaky Integrator Neurons and CTRNNs Dr. Kenneth Stanley October 25, 2006

2. Artificial Neurons are a Model • Standard activation model • But a real neuron doesn’t have an activation level • Real neurons fire in spike trains • Spikes/second is a rate • Therefore, standard activation can be thought of as outputting a firing rate at discrete timesteps (i.e. rate encoding) Wolfgang Maass, http://www.tu-graz.ac.at/igi/maass

3. What is Lost in Rate Encoding? <> • Timing information • Synchronization • Activity between discrete timesteps 30 Neurons Firing in a monkey’s striate cortex From Krüger and Aiple [Krüger and Aiple, 1988]. Reprinted from www.igi.tugraz.at/ maass/123/node2.html

4. Spikes Can Be Encoded Explicitly • Leaky integrate and fire neurons • Encode each individual spike • Time is represented exactly • Each spike has an associated time • The timing of recent incoming spikes determines whether a neuron will fire • Computationally expensive • Can we do almost as well without encoding every single spike?

5. Yes: Leaky Integrator Neurons (CTRNNS; Continuous Time Recurrent Neural Networks) • Idea: Calculate activation at discrete steps but describe rate of change on a continuous scale • Instead of activating only based on input, include a temporal component of activation that controls the rate at which activation goes up or down • Then the neuron can react to changes in a temporal manner, like spikes

6. Activation Rate Builds and Decays Input to neuron • Incoming activation causes the output level to climb over time • We can sample the rate at any discrete granularity desired • A view is created of temporal dynamics without full spike-event simulation Activation Level (i.e. spike rate) Output over time time

7. What is Leaking In a Leaky Integrator? • The neuron loses potential at a defined rate • Each neuron leaks at its own constant rate • Each neuron integrates at the same constant rate as well Leaking activation level (membrane potential) Activation Level (i.e. spike rate) time

8. Leaky Integrator Equations Leak • Expressing rate of change of activation level: • Apply Euler Integration to derive discrete-time equivalent • Expressing current activation in terms of activation on previous discrete timestep: Real time Between steps Equations from: Blynel, J., and Floreano, D. (2002). Levels of dynamics neural controllers. In Proceedings of the Seventh International Behavior on From Animals to Animats, 272–281.

9. What Can a CTRNN Do? • With the right time constants for each neuron, complex temporal patterns can be generated • That is, the time constants are a new parameter (inside nodes) that can evolve • More powerful than a regular RNN • Capable of generating complex temporal patterns with no input and no clock

10. Pattern Generation for What? • Walking gaits with no input! Evolution of central pattern generators for bipedal walking in a real-time physics environmentT Reil, P Husbands - Evolutionary Computation, IEEE Transactions on, 2002

11. Reil and Husbands Went on to Found the Company NaturalMotion

12. Pattern Generation for What? • Salamander walking gait • Wing flapping Ijspeert A.J.: A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander, Biological Cybernetics, Vol. 84:5, 2001, pp 331-348. Evolution of neuro-controllers for flapping-wing animats - group of 2 »JB Mouret, S Doncieux, L Muratet, T Druot, JA … - Proceedings of the Journees MicroDrones, Toulouse, 2004

13. Maybe Good for Other Things with Temporal Patterning • Music? • Tasks that we typically do not conceive in terms of patterns? • Learning tasks (better than a simple RNN?; Blynel and Floreano 2002 paper) • Largely unexplored • How far away from the benefits of a true spiking model?

14. Leaky NEAT • There is a rough, largely untested leakyNEAT at the NEAT Users Group files section: • http://groups.yahoo.com/group/neat/files/ • Introduces a new activation function and new time constant parameter in the nodes • A new leaky-rtNEAT will soon be available too • The topology of most CTRNNs in the past was determined completely by the researcher

15. Next Topic: Non-neural NEAT, Closing Remarks on Survey Portion of Class • Complexification and protection of innovation in non-neural structures • Example: Cellular Automata neighborhood functions • What have we learned, what is its significance, and where does the field stand? Reading: Mitchell Textbook pp. 44-55 (Evolving Cellular Automata) think about: How would NEAT apply to this task?