Evolutionary Robotics

1. Evolutionary Robotics Gabriela Ochoa

2. Robotics - Introduction • Robots in the movies (C3P0, Terminator): fantastic, intelligent, even dangerous forms of artificial life • Robots of today are not walking, talking intelligent machines • Today, we find most robots working for people in factories, warehouses, and laboratories • In the future, robots may show up in other places: our schools, our homes, even our bodies.

3. Mainstream Robotics • Most robots: a helping hand. Help people with tasks that would be difficult, unsafe, or boring for a real person • A robot have 5 main components • Controller • Arm • Drive • End-Effector • Sensor

4. Autonomous Robotics • Robot: A versatile mechanical device equipped with effectors and sensors under the control of a computing system • Human: eyes, ears (sensors); hands, legs, mouth (effectors) • Robot: cameras, infrared range finders (sensors); various motors (effectors) • Autonomous Robots: those that make decisions on their own, guided by the feedback they get from their physical sensors

5. Classic AI Approach PERCEPTION REPRESENTATION IN WORLD MODEL -- REASONING ACTION Divide & Conquer: a complex problem is decomposed into separate, less daunting subproblems Clasical approaches tu robotics, assume a primary decomposition into Perception, Planning and Action Modules

6. Subsumtion Architecture (Rodney Brooks) • Problems with the classical approach: • It is not clear how a robot control system should be decomosed • Complex Interactions between subsystems, not only direct links, also mediated via the environment • Subsumtion Architecture: slow and and careful building up of a robot control system layer by layer • Get something simple working (debugged) first • Then try and add extra 'behaviours'

7. The Evolutionary Approach • Brooks' subsumption architecture is 'design-by-hand', but inspired by an incremental, evolutionary approach • Alternative: explicitly use evolutionary techniques to incrementally evolve increasingly complex robot control systems • When evolving robot 'nervous systems' with some form of GA, then the genotype ('artificial DNA') will have to encode: • The architecture of the robot control system • Also maybe some aspects of its body/motors/sensors

8. Evolutionary Robotics The use of evolutionary (genetic) algorithms to develop `artificial nervous systems' for robots. ER can be done • for Engineering purposes - to build useful robots • for Scientific purposes - to test scientific theories It can be done • for Real or • in Simulation

9. Approaches to Evolutionary Robotics • R. D. Beer: Agent control using NNs (continuous-time recurrent NNs) Tasks: Chemotaxis, locomotion control 6-legged insect-like • M. Colombeti, M. Dorigo: Classifier Systems • D. Floreano: Kephera robots, simple recurrent NNs • J. Koza, C. Raynolds: GP to develop S. Arch. • R. Watson: Embodied Evolution • I. Harvey, P. Husband U. Sussex: continuous-time recurrent NNs CTRNN

10. Sussex Approach to ER • Robot as a whole: body, sensors, motor and control system (or “nervous system”) – as a dynamical system coupled with a dynamic environment • Continuos time recurrent NN, with temporal delays on links • Genotypes: specify the architecture of the NN

11. A top-down view of the simulated agent, showing bilaterally symmetric sensors (“photoreceptors”). Reversing the sensor connections will initially have a similar effect to moving the light source as shown.

12. Dynamic Recurrent Neural Networks DRNNs • DRNNs, or CTRNNs • Convenient way of specifying a class of dynamical systems • Different genotypes will specify different DSs, giving robots different behaviours. This is just ONE possible DRNN, which ONE specific genotype specified.

13. DRNN Basics The basic components of a DRNN are these (1 to 4 definite, 5 optional)

14. NNs Equations CTRNNs (continuous-time recurrent NNs), where for each node (i = 1 to n) in the network the following equation holds: yi = activation of node i i = time constant, wji = weight on connection from node j to node i (x) = sigmoidal = (1/1+e-x) • i= bias, Ii = possible sensory input.

15. Genotypes • Finite number of nodes • Thresholds or details of non linear activatation function • Links and connections between nodes: sepecifying weights and time-delays on the links • Optional: Weight-changing rules • Subset of the nodes, designed as input nodes, receiving sensory inputs • Output or motor nodes • Other nodes (“hidden”) arbitrary number

16. Genetic Encoding Scheme

17. Example of an Experiment • Simulation: round, two-wheeled, mobile robot with touch sensors and two visual sensors • Task: robot need to reach the centre of a simulated circular arena (white walls and black floor and ceiling) • Fitness function: • robots are rated on the basis of how much time they spent at or near the centre. • Measuring the distance d of the robot from the centre, and weighting this distance by a Gaussian G • Over 100 time steps, G is summed to give a final score • Robot starts near the perimeter, facing in a random direction

18. Evaluating a robot When you evaluate each robot genotype, you • Decode it into the network architecture and parameters • Possibly decode part into body/sensor/motor parameters • Create the specified robot • Put it into the test environment • Run it for n seconds, scoring it on the task. Any evolutionary approach needs a selection process, whereby the different members of the population have different chances of producing offspring according to their fitness

19. Results Typical path of a succesfully evolved robot wich heads fairly directly for the centre of the room and circles there, using input from 2 photoreceptors. The direction the robot is facing is indicated by arrows for each time step

20. Real Robot vs. Simulation Evaluate on a real robot, or Use a Simulation ? On a real robot it is expensive, time-consuming -- and for evolution you need many many evaluations. On a simulation it should be much faster BUT AI (and indeed Alife) has a history of toy, unvalidated simulations, that 'assume away' all the genuine problems that must be faced. Eg: grid worlds "move one step North“, Magic sensors "perceive food"

21. Principled Simulations ? How do you know whether you have included all that is necessary in a simulation? -- only ultimate test, validation, is whether what works in simulation ALSO works on a real robot. How can one best insure this, for Evolutionary Robotics ? Noise: put an envelope-of-noise, (through variations driven by random numbers), around crucial parameters whose real values you are unsure of. “Evolve for more robustness than strictly necessary"