Spacetime Constraints Revisited

1. Spacetime Constraints Revisited Joe Marks J. Thomas Ngo Using genetic algorithms to find solutions to spacetime constraint problems in 2D

2. Difficulties with SC problems • Multimodality • exponential number of possible solutions • Search space discontinuities • small changes in actuators leads to large changes in trajectory

3. Why Genetic Algorithms? • Traditional SC algorithms • Initial trajectory needed to work on • Initial trajectory is done by human hand - “coarse” • Initial trajectory is locally optimized • Genetic Algorithms • Discovers solutions “from scratch” • Cover global optimization problem • Can find “novel” solutions that were not imagined

4. Overview of GA in SC problems • All trajectories are described as behaviors • Behaviors generated by parametric algorithms (nothing smart here) • Evolutionary computation chooses behavior parameters to find optimal solutions • can find multiple effective solutions • Better solutions (genomes) bred further • Worse solutions bred-out of gene pool

5. GA algorithm for 2D simulation(Ngo and Marks) • Dynamics module • simulates a physics driven world for testing creatures • Behavior module • generates behavior using parameterized algorithm • Search module • uses GA to choose values for parameters • generates near-optimal solutions

6. Dynamics Module • Most CPU intensive part of this algorithm • Simulating simplified physics • Based on work by Hahn • rate of change of internal degrees of creature controlled, rather than calculating torques • friction and slippage simulated upon contact

7. Behavior Module • Behaviors generated by stimulus-response (SR) control algorithm • causes instinctive reflexes to conditions • conditions are stimulus functions over senses • reflexes are responses to conditions • avoids traditional use of forces • behaviors selected in search module • no learning or planning algorithms involved

8. Stimulus functions • Scalar functions based on sense variables • state of joint angles • force between body endpoints and floor • vertical velocity of center of mass • height from floor of center of mass • Contain parameters that are determined by GA-based search module • Stimulus functions exhibit sensitive regions • locus of points for which function is positive • important notion during mutation

9. Response • Change in shape in reaction to a condition • Conducted on highest valued positive stimulus function • Change of creature’s actual shape to a target shape • Change kept smooth by damped motion equations • Target shape may change during a response • Creature must respond to another stimulus • Response is active for several time steps

10. Search Module • Use of GA to pick near optimal behaviors • Parallel processing of solutions • each processor handles one genome solution per generation do parallel Randomize genome end do for generation = 1 to number_of_generations do parallel Evaluate genome Select mate from another processor Cross genome with mate Mutate genome end do end for

11. Randomization • Initial parameters chosen at random • Favorable initial gene pool formed • hill climbing algorithm used to find good initial solutions to populate gene pool • each solution mutated, re-evaluated four times • mutation skewed towards multi-step solutions • best of five solutions on each processor used • final population is non-random, skewed

12. Evaluation • One of the most important aspects of GA • Net horizontal distance covered by center of mass in given time • Sometimes this encourages “cheating” • e.g., leap head-first rather than walking • To encourage walking, modify criteria • e.g., net horizontal distance covered by midpoint between two feet - i.e., use your feet!

13. Mate Selection • Only local mating permitted • maintains diversity, handles multimodality • mate chosen on a random walk of 10 steps • if better solution than self found, mate is chosen • otherwise, current solution stays single • produces large local colonies of good genes • larger colonies use up more processing power • Convergence - when one colony dominates

14. Crossover • Linear crossover not seen as meaningful • certain parameters should migrate as groups • More structured, specialized crossover used • two (of ten) SR pairs taken from self • six (of ten) SR pairs taken from mate • one SR pair created with stimuli and response taken from each parent, respectively • one SR pair created with numbers taken by random from parents

15. Mutation • Mutation algorithm tailored for SR method • One SR pair in genome is subject to creep • each parameter in pair is slightly changed • One SR pair is randomized from scratch • but one corner of sensitive regions of new stimulus function coincides with original sense-space trajectory • keeps new functions from either dominating or not having any effect.

16. Results • Hardware, software used • 4,096 processor Thinking Machines CM-2 • Code written in C* • Creatures used • 5 rod figures; 50 time steps; 100 generations • Five-rod Fred • Mr. Star-man • Beryl Biped • importance of adapting criteria • 30 to 60 minutes of computation used