CAP6938 Neuroevolution and Developmental Encoding Non-Neural NEAT and Closing Remarks

1. CAP6938Neuroevolution and Developmental Encoding Non-Neural NEAT andClosing Remarks Dr. Kenneth Stanley October 30, 2006

2. Outline • Complexification is a general concept • Protecting innovation is a general concept • Therefore, they can apply to anything without a defined dimensionality • Example: Cellular Automata

3. Complexification is a General Concept • Solving a smaller version of a problem and expanding the solution • Making a rough estimate and refining it • Building a structure piece by piece • Elaboration of a pre-existing concept

4. Complexification Does Not Mean Optimizing Random DimensionsFrom a Set • Example: 10-dimensional search space • Now hold d2 through d10 constant and search d1 • Once you get a good value for d1, start searching both d1 and d2 together, and so on • This is not complexification • It is a naïve search assuming independent variables • Subject to simple deception • Usually won’t work

5. Then What Does it Mean? • Complexification means increasing information about the solution • (Optimizing d1 does not increase general information about the solution) • Initial dimensions are a complete solution on their own (nothing is held at zero) • Complexification means finding the dimensionality of the solution is part of the problem • A neural network can have any number of weight dimensions and solve the same problem • Most “dimensions” outside the current structure have no meaning on their own

6. Example • 3 dimensions • Is dimension 2372 held at zero? • What exactly is dimension 2372? • It depends on how the other 2371 dimensions turn out to relate to each other • It is undefined; it doesn’t exist: • Not like d10 in prior example, which always exists • Complexification is searching infinite undefined dimensions, or rather, it is not performing search in the usual sense. It is increasing information. 1 3 2

7. Example 2 y • Problem: Find an expression of this function • Complexification says start with a very low dimensional approximation as accurate as possible in its space • Red line: 2-dimensional estimate y=mx+b • Now we could add new terms and refine the estimate • Analogous to bending the line like a rubber band for each new dimension added • New estimate does not necessarily need exactly the same term “mx+b” x

8. Protecting Innovation is a General Concept • New ideas need time to mature • Children need time to grow up • Ph.D. students need room to make mistakes • Bigger often means slower, but not stupider • Einstein was not the teacher’s pet • The long run is what matters • If we kicked him out early, we’d all lose

9. Speciation Protects Innovation • An “idea” is represented as a niche • The niche is a local, protected competition • One niche does not directly compete with another • Only the absolute worst are purged after sufficient opportunity is spent

10. General Concepts Means They Don’t Have to Apply to a Neural Network • Complexification and protection of innovation go hand in hand • In order to elaborate, one must protect potential elaborations • In order to grow one must have room

11. Novel Phenotype: Cellular Automata • Set of pixels that change over time according to neighborhood rules • The Game of Life is a familiar example of 2D cellular automata From: http://www.bitstorm.org/gameoflife/

12. 2D Cellular Automata • Pixels are in a line instead of a plane • Change over time can be represented as a vertical graph: time From: Melanie Mitchell, James P. Crutchfield, and Rajarshi Das, "Evolving Cellular Automata with Genetic Algorithms: A Review of Recent Work", In Proceedings of the First International Conference on Evolutionary Computation and Its Applications (EvCA'96), Russian Academy of Sciences (1996).

13. Neighborhood Rules • Next state for pixel determined by pixels in its neighborhood within some radius: 2(2r+1) bits per rule table From: Melanie Mitchell, James P. Crutchfield, and Rajarshi Das, "Evolving Cellular Automata with Genetic Algorithms: A Review of Recent Work", In Proceedings of the First International Conference on Evolutionary Computation and Its Applications (EvCA'96), Russian Academy of Sciences (1996).

14. Can It Do Anything Useful? • Maybe it can compute functions • Popular task: Fill the line with whichever color is in the majority (Density Classification) • Successful attempts: (r=3; 128 bits/genome)

15. Assessing Performance • Measure % correct over unbiased distribution of many initial conditions • Best performance is 86% on 149 pixels with r=3 (Juillé and Pollack 1998) using coevolution of rules and initial conditions • Could we do a lot better than 86%? • Maybe with complexification

16. Complexifying Cellular Automata • How?

17. Complexifying Cellular Automata • How? Expand the neighborhood • Neighborhood doesn’t need to be symmetric or even contiguous • Is this really complexification? • Yes: Unexpressed dimensions are undefined without knowing all the dimensions • The initial rules give us only a little information, but good information • The dimensions of the search space are the bits of the rule, not the neighborhood positions • The rule includes the neighborhood positions, i.e. there is structure. Position is a historical marker in this case. 00 0 01 0 10 1 11 0 0 0 1 1

18. Even More Abstract Complexification Solution • A very wide neighborhood could be input into a neural network that computes a function of those inputs and outputs the next value for the bit in the middle • The network that computes the function can complexify Evolved Topology … … …

19. Conclusion • Complexification and protection of innovation may allow more complex and therefore powerful neighborhood functions to evolve (maybe beat 86% by using with coevolution?) • Complexification and protection of innovation may allow far more complex solutions to anything

20. NE & DE: What Have We Learned • Search is not just optimization • Expanding complexity over generations is a powerful idea • Protecting innovation is as well • Neural networks can be grown with this method • The mapping between genotype and phenotype is equally important: • Reuse of genes is powerful • The neural model can be enhanced in several ways • NEAT can evolve any kind of structure, including DE/indirect encodings, and CPPNs

21. What Is Its Significance? • These are the forces of nature • We are unlocking nature’s box by understanding the underlying algorithms • Much of the beauty and complexity of nature, and civilization itself, resulted from these simple processes • Biological evolution was an unguided process • What will we create if we take its reigns and guide it?

22. Where is the Field? • Two parallel streams • How to evolve with complexification • How to represent with reuse (DE) • More progress so far on 1 than 2 • Indirect encoding is on the brink • The two streams are merging (e.g. CPPNs) • A complexifying system with an efficient encoding (mapping) is the next generation system

23. Next Topics:Technical topics in implementing complexifying evolutionary systems and presenting results. • Practical implementation issues • Questions and group discussion/problem solving • How to present research results Reference: Section 3 of Real-time Neuroevolution in the NERO Video Game by Stanley, Bryant, and Miikkulainen (2005). (Has most up-to-date NEAT description)