CAP6938 Neuroevolution and Artificial Embryogeny Working with NEAT

1. CAP6938Neuroevolution and Artificial EmbryogenyWorking with NEAT Dr. Kenneth Stanley February 8, 2006

2. Exploiting Speciation • Keep a list of species with increasing ID’s • Species elitism: duplicate the species champs in the next generation • Only if species is above size 4 or 5 • Irrelevant in rtNEAT • Stagnation Detection • Track generations since last improvement for each species • Drop fitnesses of stagnant species to near 0

3. Crossover Issue • Two different connections can end up connecting the same nodes • Maybe disallow 5 9

4. Look Out for Bugs • Gene pointing to nodes that aren’t in Gemome • Gene innovation numbers out of order • Node innovation numbers out of order • Duplicate genes • May want to check for these situations after major operations during debugging • See Genome::verify() in NEAT C++ • Can check whole population in a loop

5. Network Activation • Use standard arbitrary-topology activation • (Lecture 2) • Do not activate in layers!

6. Real-time Evolution • If all your agents are evaluated simultaneously, rtNEAT may be more appropriate • rtNEAT produces and replaces one individual at a time (no generations) • See “REAL-TIME NEUROEVOLUTION IN THE NERO VIDEO GAME ” on my homepage • Also see NERO • rtNEAT source coming soon

7. Your Genes May Have Extra Fields • Time constants in nodes • Activation function in nodes • Learning parameters in connections • You can have whatever you want, whatever you are evolving • Should extra params affect compatibility? • Remember, more fields means more possibilities, but also more dimensions

8. Important Methods • Crossover: mate_multipoint and mate_multipoint_avg • Mutation: mutate_add_connection, mutate_link_weights, mutate_enable_bits, other? • Special Genomes Operations: duplicate, compare (getting the delta for compatibility distance) • reproduce (inside Species) • epoch (inside Population) • Remember: http://www.freshraisins.com/neatdoc/files/genetics-cpp.html

9. Artificial Embryogeny and Non-neural NEAT • Your genes may not map directly to pieces of structure • However, genes can still be marked with historical markings and complexification can still occur • Need a standard starting structure

10. Sensors Must Be Chosen Wisely

11. Noisy Evaluation • Randomness in an environment can cause evaluation to be inconsistent • For example, in a game of chance • Same individual can receive different fitness on different trials • Causes confusion for evolution • Not always fatal, but merits attention • Maybe give all individuals in same generation same evaluation sequence

12. Saving and Loading /* Species #3 : (Size 5) (AF 6.03094) (Age 29) */ /* Organism #0 Fitness: 8.77021 Error: 1.03855 */ genomestart 0 trait 1 0.1 0 0 0 0 0 0 0 trait 2 0.2 0 0 0 1 0 0.21903 0 trait 3 0.65 0.110145 0 0 0 0 0 0 node 1 3 1 3 node 2 3 1 1 node 3 3 1 1 node 4 1 0 2 node 9 1 0 0 node 86 1 0 0 gene 3 1 4 -0.798127 0 1 -0.798127 1 gene 3 2 4 3 0 2 3 1 gene 2 3 4 -3 0 3 -3 1 gene 2 3 9 0.505339 0 12 0.505339 1 gene 1 9 4 1.02135 0 13 1.02135 1 gene 1 1 9 -3 0 95 -3 1 gene 1 2 9 2.28649 0 266 2.28649 0 gene 1 2 86 -1.22499 0 281 -1.22499 1 gene 1 86 9 1.72795 0 282 1.72795 1 genomeend 0 /* Organism #1 Fitness: 5.37326 Error: 1.68197 */ genomestart 1 trait 1 0.1 0 0 0 0 0 0 0 trait 2 0.2 0 0 0 1 0 0.21903 0 trait 3 0.65 0.110145 0 0 0 0 0 0 node 1 3 1 3 node 2 3 1 1 node 3 3 1 1 node 4 1 0 2 node 9 1 0 0 • Genomes/Populations should be savable and loadable • Important for debugging • Very disappointing to lose a good thing forever • Some people use standards like XML

13. Modularity Means Future Expansion (and maybe other users) • Think about what you might want to do someday • Add new parameters to nodes? • Add new parameters to links? • Change the mapping? • Change platforms? • Completely different experiments? • More visualization? • Change phenotype? • Change genetyoe?

14. Too Much Scope = Too Much Rope • Go for proof of concept • What feature matters least? • If nothing works what could be a prerequisite task? • Incremental evolution strategies

15. Next Class: Artificial Embryogeny • The power of reuse • Prior work • Biological underpinnings • The future A Taxonomy for Artificial Embryogeny by Kenneth O. Stanley and Risto Miikkulainen (2003) Homework due 2/15/05: Working domain and phenotype code. Turn in summary, code (if too long just include headers and put rest on web), and examples demonstrating how it works.

16. Project Milestones (25% of grade) • 2/6: Initial proposal and project description • 2/15: Domain and phenotype code and examples • 2/27: Genes and Genotype to Phenotype mapping • 3/8: Genetic operators all working • 3/27: Population level and main loop working • 4/10: Final project and presentation due (75% of grade)