1 / 96

The Neuronal Replicator Hypothesis

The Neuronal Replicator Hypothesis. Chrisantha Fernando & Eors Szathmary CUNY, December 2009 1 Collegium Budapest (Institute for Advanced Study), Budapest, Hungary 2 Centre for Computational Neuroscience and Robotics, Sussex University, UK

carl
Download Presentation

The Neuronal Replicator Hypothesis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The Neuronal Replicator Hypothesis • Chrisantha Fernando & Eors Szathmary • CUNY, December 2009 • 1Collegium Budapest (Institute for Advanced Study), Budapest, Hungary • 2Centre for Computational Neuroscience and Robotics, Sussex University, UK • 3MRC National Institute for Medical Research, Mill Hill, London, UK • 4Parmenides Foundation, Kardinal-Faulhaber-Strase 14a, D-80333 Munich, Germany • 5Institute of Biology, Eötvös University, Pázmány Péter sétány 1/c, H-1117 Budapest, Hungary

  2. Visiting Fellow MRC National Institute for Medical Research London Post-Doc Center for Computational Neuroscience and Robotics Sussex University Marie Curie Fellow Collegium Budapest (Institute for Advanced Study) Hungary

  3. The Hypothesis • Evolution by natural selection takes place in the brain at rapid timescales and contributes to solving cognitive/behavioural search problems. • Our background is in evolutionary biology/the origin of non-enzymatic template replication/evolutionary robotics/computational neuroscience.

  4. Outline • Limitations of some proposed search algorithms, e.g. • Reward biased stochastic search • Reinforcement Learning • How copying/replication of neuronal data structures can alleviate these limitations.

  5. Mechanisms of neuronal replication • Applications and future work

  6. Simple Search Tasks • Behavioural and neuropsychological learning tasks can be solved by stochastic-hill climbing • Stroop Task • Wisconsin Card Sorting Task (WCST) • Instrumental Conditioning in Spiking Neural Networks • Simple inverse kinematics problem

  7. Stochastic Hill-Climbing • Initially P(xi = 1) = 0.5, Initial reward = 0 • Make random change to P • Generate M examples of binary strings • Calculate reward • If r(t) > r(t-1), keep changes of P, else revert to previous P values. • One solution, change solution, keep good changes, loose bad changes.

  8. Can get stuck on local optima

  9. Stroop Task GreenRedBlue PurpleBluePurple BluePurpleRedGreenPurpleGreen Name the colour of the words.

  10. dW = Reward x pre x post • Decreased reward -> Instability in workspace Dehaene et al, 1998

  11. WCST • Each card has several “features”. Subjects must sort cards according to a feature (color, number, shape, size).

  12. Rougier et al 2005. PFC weights stabilised if expected reward obtained, destabilised if expected reward not obtained, i.e. TD learning

  13. Instrumental Conditioning

  14. In a spiking neural net • Simple spiking model • Random connections • STDP • Delayed reward • Eligibility traces • Synapse selected Izhikevich 2007

  15. Simple spiking model

  16. STDP

  17. Time tpre

  18. Time tpost

  19. Interval = Tpost - Tpre

  20. Time tpost

  21. Time tpre

  22. Interval = Tpost - Tpre

  23. A simple 2D inverse kinematics problem

  24. Reinforcement Learning • For large problems a tabular representation of state-action pairs is not possible. • How does compression of state representation occur? Function approximation • Domain-specific knowledge provided by the designer, e.g. TD-Gammon was dependent on Tesauro’s skillful design of a non-linear multilayered neural network, used for value function approximation in the Backgammon domain consisting of approximately 1020 states” p20 [51].

  25. So far… • SHC works on simple problems • RL is a sophisticated kind of SHC • In order for RL/SHC to work, action/value representations must fit the problem domain. • RL doesn’t explain how appropriate data-structures/representations arise.

  26. Large search space so random search or exhaustive search not possible. Representation critical local optima. Requires internal sub-goals, no explicit reward. • What neural mechanisms underlie complex search?

  27. What is natural selection? • multiplication • heredity • variability Some hereditary traits affect survival and/or fertility

  28. Natural selection reinvented itself

  29. Evolutionary Computation • Solving problems by EC also requires decisions about genetic representations • And about fitness functions • For example, we use EC to solve the 10 coins problem

  30. Fitness function • Convolution of desired inverted triangle over grid • Instant fitness = number of coins occupying he inverted triangle template • An important question is how such fitness functions (subgoals/goals) could themselves be bootstrapped in cognition.

  31. Michael Ollinger, Parmenides Foundation, Munich

  32. Structuring Phenotypic Variation • Natural Selection can act on • genetic representations • variability properties (genetic operators, e.g mutation rates)

  33. Variation in Variability A Improvement of representations for free…

  34. B

  35. Non-trivial Neutrality g1 ed 1 ed 2 p g2 Adapted from Toussaint 2003

  36. Population Search • Natural selection allows redistribution of search resources between multiple solutions. • We propose that multiple (possibly interacting) solutions to a search problem exist at the same time in the neuronal substrate.

  37. A B C D B A A D C A B C D B A A D C

  38. A B C D B D B A D A A A B C D C C Waste A B C D D’ D’’ D A D’’’ D’ D’’ D’’’ D

  39. Can units of selection exist in the brain? • We propose 3 possible mechanisms • Copying of connectivity patterns • Copying of bistable activity patterns • Copying of spatio-temporal spike patterns & explicit rules

  40. Copying of connectivity patterns

  41. How to copy small neuronal circuits DNA neuronal network

  42. STDP and causal inference

More Related