Adaptive Robotics

1. Adaptive Robotics • COM2110 • Autumn Semester 2008 • Lecturer: Amanda Sharkey

2. “robots” in the news • Loebner test at Reading • Winning chatbot ‘Elbot’ convinced 3/12 judges it was human in 5 minute chat • Explicitly referred to itself as a machine • "Hi. How's it going?" one judge began. • "I feel terrible today," Elbot replied. "This morning I made a mistake and poured milk over my breakfast instead of oil, and it rusted before I could eat it."

3. Last week Animal behaviour: reflex, taxis, fixed action patterns (Loeb) Behaviorism Grey Walter and Elsie and Elmer - Earlier robots, “electric dog” or “artificial heliotrope machine” Braitenberg vehicles Brooks’ subsumption architecture Reactive-deliberative continuum of robot control

4. Some themes so far …. Emergent behaviour – interaction of decentralised behaviours - interaction with the environment Embodiment and situatedness - importance of looking at direct unmediated interface with the environment (sensors) Simplicity - keep it simple Biological inspiration

5. Behaviour-based control • Fixed hierarchy of behaviours • Coordination of behaviours via subsumption mechanism – triggered by the environment • Emergent global behaviour as a result of interaction of component behaviours • Design of behaviours – depends on human designer • Q: how to arrive at good decomposition of behaviours to accomplish a task? • Implementation: Behaviours could be programmed as condition-action rules

6. Meaning of Adaptive in Adaptive Robotics? • Adaptation in biological systems • Learning • Evolution – adaptation to environment across generations • Adaptive behaviour does not necessarily need to involve learning within the lifetime of the individual

7. Innate behaviour in animals/insects • Innate: instinct and genes determine behaviour • Innate behaviours have evolved (phylogenetic learning) • Phylogenetic (evolutionary) learning vs ontogenetic (during life) learning • Also called Nature/Nurture issue • Can also combine innate behaviours with some learning

8. Innate behaviour and environment • See coastal snail from last week • Close fit between innate reactive behaviours and the environment • Combination of taxis (negative and postive phototropism, and negative geotaxis) leads to food and survival

10. Bats and noctoid moths • Moths respond to ultrasound emissions from bats • Loud sound – fly away, • Soft sound – tumble unpredictably (wing beats desynchronised) • Two tympanic membranes (ears) on moth body attuned to bat emissions. Connected to wings, and producing different behaviour depending on loudness. • Example of reflex behaviour: a highly adaptive perceptual mechanism.

11. Innate behaviour • Instinct • Often characterised by Fixed Action Pattern (FAP) • (all or none response) • Sign stimulus to release FAP • E.g. detection of ultrasound from bats by prey moths

12. Innate behaviour cont. • Reflexes • Simple responses to simple behaviour • E.g. insect flight response, when legs not touching the ground • Taxis: a directed movement towards (or away from) a stimulus • Phototaxis (light) • Geotaxis (gravity) • Phonotaxis (sound)

13. Interaction between simple reflexive behaviours and the environment can lead to apparently sophisticated behaviour

14. Innate behaviour in humans • Innate interest in human faces, or face-like stimuli

15. Innate + Learning • Innate behaviour that requires initialisation • E.g. Imprinting and critical period • The biologist Konrad Lorenz • Rapid bond formation during critical period • E.g. greylag geese

17. Forms of adaptation • Evolution – adaptation over generations • Habituation – getting used to a stimulus. (e.g. birds come to ignore stuffed owl) • Associative learning • Classical conditioning • Operant conditioning • Imitation learning • Insight • Complex behaviour • Tool use • Language use

18. Classical conditioning • Initially researched by Pavlov • Unconditioned stimulus (US) e.g. food in mouth • Unconditioned response (UR) e.g. salivating • Conditioned stimulus (CS) e.g. bell • Conditioned response (CR) e.g. salivating

22. Operant conditioning • The probability of making a response is increased by rewarding it. • Best known example: training a rat to press a lever to obtain a food reward.

23. Learning in insects? • Instincts: stereotyped behaviour • Learning: changes in behaviour as a result of experience • Types of learning: • Habituation (sensory adaptation) • Associative (classical conditioning) e.g bees trained to find food in particular coloured dishes • Instrumental (operant conditioning): learning associated with positive/negative consequences. E.g. crickets learning which of two holes were escape routes • Latent learning: associative learning without reward – e.g. bees learning landmarks to find nest • Insight – e.g. tool use

24. Insects can learn • But sophisticated behaviour of swarms of bees or colonies of ants mostly results from the interactions of fixed reactive behaviours of ants, with each other and the environment. • Simple behaviours – interaction with world – emergence of complex behaviour.

25. Behaviour with underlying simple mechanism can appear more complex • People often invent more complex explanations • Can be exploited in robotics

26. Beast1: A simple Braitenberg vehicle

27. Distal v Proximal descriptions Distal descriptions are items such as hide, approach, attack etc Proximal description table for beast1

28. Valentino Braitenberg

30. Pre-designed controller • Braitenberg-type controller to perform straight motion and obstacle avoidance with Kephera robot • Positive connection between wheel and sensors on its own side: rotation speed of wheel proportional to activation of sensor • Negative connection between wheel and sensors on the opposite side: rotation speed of wheel is inversely proportional to sensor activation • Positive offset value to each wheel generates forward motion

31. Positive connection between wheel and sensors on its own side: rotation speed of wheel proportional to activation of sensor Negative connection between wheel and sensors on the opposite side: rotation speed of wheel is inversely proportional to sensor activation

32. Weighted sum of incoming signals steers robot away from objects • But design needs prior knowledge of sensors, motors and environments • E.g. if sensor has lower response profile than other sensors, its outgoing connections require stronger weights.

33. Robot news 2

34. Recent News: Robot suit for walking in • HAL - short for "hybrid assistive limb" – is a computerized suit with sensors that read brain signals directing limb movement through the skin.The 22 pound (10 kilogram) battery-operated computer system is belted to the waist. It captures the brain signals and relays them to mechanical leg braces strapped to the thighs and knees, which then provide robotic assistance to people as they walk.Cyberdyne, a new company in Tsukuba outside Tokyo, will mass-produce HAL. • Daiwa House Industry Co. will lease HAL suits to Japanese care facilities for the elderly and others for those with disabilities. It plans to rent 500 units over the next year.

36. Boffins unveil life-like robogirl • THIS is the most life-like robot suit ever – the cyber girl Repliee R-1. • It has 50 sensors and a series of motors to help it move and has been built to help pensioners and disabled people move better. • Japanese boffins from Tsukuba University developed the Hybrid Assistive Limb (HAL) suit using flexible silicon skin. • And robotics company Cyberdyne Inc are set to start making it on a mass scale on Friday.

37. Advantages of Neural Nets for robotics • They provide a straightforward mapping between sensors and motors • They are robust to noise (noisy sensors and environments) • They can provide a biologically plausible metaphor • They offer a relatively smooth search space (gradual changes in weights = gradual changes in behaviour)

38. Artificial Neural Networks • ANNs can be used for control system of robots. • Can be used with fixed weights • Like inherited, innate knowledge in living beings • But how to arrive at the weights? • Trial and error? • Learning methods • Evolutionary methods

39. McCulloch and Pitt’s neurons • How can a mind emerge from the intricate chemical complexity and neural connectivity of the human brain. Warren McCulloch

40. The problem grows • Not only is there the problem of 1011 neurons. • The neurons can have up to 10,000 connections to other neurons like the motor neuron shown on the right.

41. A logical calculus of ideas immanent in the nervous system • McCulloch met Pitts in 1943 and within months they had drafted a paper on the problem that McCulloch had been trying to solve for many years. • "A Logical Calculus of Ideas Immanent in Nervous Activity" (1943). Walter Pitts

42. ---an aside • Walter Pitts • Brilliant eccentric – met McCulloch as a homeless person without income • McCulloch invited him to his home • Began researching with McCulloch, Wiener, Lettvin • Eventually McCulloch fell out with Wiener, (Mrs Wiener?) and research disbanded. Pitts fell into a decline.

43. How does a mind emerge from a brain? • McCulloch and Pitts turned to George Boole’s Language of Thought- what every computer scientist now knows as Boolean logic.

44. The solution came with the notion of ‘all or none’ neural firing.

45. “All or none” neural firing translates in binary elements • Once McCulloch and Pitts had the idea of binary threshold elements, the next step was to map onto Boolean logic. • QED? – well not quite!

46. Boolean AND Input output

47. Sensor AND

48. Boolean OR Input output

49. Boolean NOT Input output

50. Exercise 1 Write out truth tables for each of the three nets below (for inputs of 11, 10, 01, 00, show outputs) 3 2 1