slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Adaptive Robotics COM2110 Autumn Semester 2008 Lecturer: Amanda Sharkey PowerPoint Presentation
Download Presentation
Adaptive Robotics COM2110 Autumn Semester 2008 Lecturer: Amanda Sharkey

Loading in 2 Seconds...

play fullscreen
1 / 67

Adaptive Robotics COM2110 Autumn Semester 2008 Lecturer: Amanda Sharkey - PowerPoint PPT Presentation

  • Uploaded on

Adaptive Robotics COM2110 Autumn Semester 2008 Lecturer: Amanda Sharkey. Researchers at Georgia Tech have built a biologically inspired robot to perform actions of service dogs Users issue verbal commands to robot, and indicate object with laser pointer.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Adaptive Robotics COM2110 Autumn Semester 2008 Lecturer: Amanda Sharkey' - harlow

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Adaptive RoboticsCOM2110

Autumn Semester 2008Lecturer: Amanda Sharkey

Researchers at Georgia Tech have built a biologically inspired robot to perform actions of service dogs
  • Users issue verbal commands to robot, and indicate object with laser pointer.
  • Eg. Fetching items, or closing doors or drawers.
  • Worked with trainers of dogs
  • Could replicate 10 commands and tasks
    • Robots could reduce costs, and waiting times for service dogs
    • Companionship?
assignment 30
Assignment (30%)
  • Due by Monday 17th Nov at 11 am
  • Write an essay (1500-2500 words) on one of the following topics. You should use the lectures as a starting point, but also research the topic yourself. Plan your answer. Include a reference section, with the references cited in full. 

1. Identify the main characteristics of Behaviour-based robotics, and contrast the approach to that of “Good old-fashioned AI”.

2. To what extent did Grey Walter’s robots, Elsie and Elmer, differ from robots that preceded, or followed them.

3. Explain how the concepts of “emergence” and “embodiment” are related to recent developments in robotics and artificial intelligence.

assignment notes
Assignment notes

Essay – Try to structure it with introduction, body and conclusion. Plan out an argument, like pseudo-code.

Include references

- in the text, either numerical [1], or by name, Cao et al (1997)

- In a Reference section at the end

journal articles:

[1] Cao, Fukunaga and Kahn (1997) Cooperative mobile robotics: antecedents and directions. Autonomous Robots, 4,1, 7-27.

And books

Gordon, D. (1999) Ants at work: How an insect society is organised. W.W.Norton and Co.,London.

Secondary citation

Sharkey, A.J.C. (1800) How to cite references, cited in Gordon, D. (1999) Ants at work: How an insect society is organised. W.W. Norton and Co. London.


Training weights with delta rule to produce target outputs

  • Testing – presenting new inputs to (already trained) weights, to see what the output is. Generalisation performance is the percentage of the test set the trained net gets right.
  • Motor outputs

Bias unit

Light sensor inputs

advantages of neural nets for robotics
Advantages of Neural Nets for robotics
  • They provide a straightforward mapping between sensors and motors
  • They are robust to noise (noisy sensors and environments)
  • They can provide a biologically plausible metaphor
  • They offer a relatively smooth search space (gradual changes in weights = gradual changes in behaviour)
example hebbian learning for collision avoidance
Example: Hebbian learning for collision avoidance
  • From Pfeiffer and Scheier (1999) On-line learning

Motor action

Collision layer

Distance sensors

Fixed weights between collision layer and motor actions.

When collision occurs, and distance sensor active, Hebbian learning used to strengthen the weight between the two.

After learning – activating the distance sensor will result in the collision detector being activated.

The robot will learn to avoid objects.

other methods of setting up ann controller
Other methods of setting up ANN controller
  • Can use delta rule (for linearly separable patterns).
  • Or can use backpropagation learning rule to train multi-layer net
  • Needs training set
  • how can you translate the behaviour of not bumping into objects into a training set?
  • See Sharkey (1998) for one method – write simple code (innate controller) for avoiding obstacles, and collect examples of inputs and outputs for training
  • Or Genetic algorithms can be used to evolve neural net controller
genetic algorithms a form of evolutionary computation
Genetic Algorithms- a form of evolutionary computation
  • A GA operates on a population of artificial chromosomes, selectively reproducing chromosomes of individuals with better performances with some random mutations.
  • Artificial chromosome (genotype) encodes characteristics of individual (phenotype)
    • E.g. could encode weights of artificial neural network
Fitness function:
  • Used to evaluate the performance of each phenotype.
  • Higher fitness values better
  • E.g. minimising difference between output of function and target
  • Smaller difference = higher fitness
Start with population of randomly generated chromosomes
  • Decode each chromosome and evaluate its fitness
  • Apply genetic operators to create new population
    • Crossover
    • Mutation
    • Selective reproduction
  • Repeat until desired individual found, or best fitness stops increasing.
genetic operators crossover and mutation
Genetic operators: Crossover and Mutation

New population created by selective reproduction.

Offspring are randomly paired, crossed over and mutated

One point crossover – for a pair of chromosomes select a random point to crossover material between two individuals.

genetic operators mutation
Genetic operators: Mutation
  • For binary representations, switch value of selected bits
  • For real value representations, increment by small number randomly extracted from distribution centred round zero.
  • Other representations: substitute selected location with symbol randomly chosen from same alphabet
genetic operators selective reproduction
Genetic operators: Selective reproduction
  • Making copies of best individuals – more copies in next generation.
  • Problem: method breaks down when all individuals have similar fitness values, (becomes like random search)
  • or one or two have higher fitness values than rest of population (they dominate)
  • Scale fitness values to enhance, or decrease individual differences
  • Rank-based selection: probability of making offspring proportional to rank
  • Truncation selection: ranking individuals and selecting top M individuals
  • Tournament based selection: randomly select 2, generate random number between 0 and 1, if number smaller than predefined parameter T, fitter individual makes offspring, or if greater, other individual reproduces.
  • Elitism: maintain best individual for next population.
Each generation:
    • fitness evaluation of all individuals in population
    • Selective reproduction
    • Crossover and mutation
  • Repeat for several generations – monitor average and best fitness: halt when fitness indicators stop increasing or satisfactory individual found.
Aim of using GAs: emergence of complex abilities from interaction between agent and environment
  • Difficult to establish what abilities and achievements are needed
  • Other approaches:
    • Incremental methods – change fitness evaluation at different stages
    • Evolution and training – can often work to both evolve and to train.
Useful reference (extensive review) -

Yao, X. (1999) Evolving artificial neural networks. Proceedings of the IEE 87, 9, 1423-1447

Possible to evolve:

  • Weights and learning parameters
  • Architectures
  • Learning rules
evolving weights and learning parameters
Evolving weights and learning parameters
  • Advantage for evolutionary robotics: don’t need to specify network response to each pattern
  • Synaptic weights encoded on genotype
  • Strings of real values, or binary values
  • GAs used to evolve the weights
  • Combining evolution with NN learning – use GAs to find initial values of weights for nets subsequently trained with backpropagation
evolving architectures
Evolving Architectures
  • Indirect coding of network on genotype – e.g. number of nodes, probability of connecting them, type of activation function
  • E.g. Harp, Samad and Guha (1989): genetic string encodes blueprint for net. Blueprint consists of several segments corresponding to a layer.
  • Segments have 2 parts: (I) node properties (no. of units, activation functions, geometric layout) (ii) outgoing properties (connection density, learning rate etc).
  • Blueprints decoded into networks, which are trained using Backpropagation.
Evolving learning rules
  • Evolvable hardware
  • E.g. Maris and Boekhorst 1996
  • Sensor position evolved
example of evolving robots
Example of evolving robots
  • Floreano, D. and Mondado, F.(1994) Automatic creation of an autonomous agent: genetic evolution of a neural network driven robot. In D.Cliff, P.Husbands, J. Meyer and S.W. Wilson (Eds) From Animals to Animats 3: Proceedings of Third Conference on Simulation of Adaptive Behaviour. Cambridge, MA: MIT Press/Bradford Books
  • Comparison of evolution of simple navigation, to predesigned architecture
Predesigned architecture:
    • Braitenberg-type controller to perform straight motion and obstacle avoidance with Khepera robot
    • Positive connection between wheel and sensors on its own side: rotation speed of wheel proportional to activation of sensor
    • Negative connection between wheel and sensors on the opposite side: rotation speed of wheel is inversely proportional to sensor activation
    • Positive offset value to each wheel generates forward motion
Weighted sum on incoming signals steers robot away from objects
  • But design needs prior knowledge of sensors, motors and environments
  • E.g. if sensor has lower response profile than other sensors, its outgoing connections require stronger weights.
evolving a controller floreano and mondado 1994
Evolving a controller(Floreano and Mondado, 1994)
  • Goal: evolving a controller to maximise forward motion while avoiding obstacles
  • Fitness function based on 3 variables
Three components in fitness function to encourage
    • Motion
    • Straight displacement
    • Obstacle avoidance

Where V is the sum of the rotation speeds of the two wheels

Δv is the absolute value of the algebraic difference between the signed speed values of the wheels

i is the normalised activation value of the infrared sensor with the highest value


First component V is computed by summing the rotation speeds of the 2 wheels (direction of rotation given by sign of read value, and speed by its absolute value).


Second component encourages 2 wheels to rotate in same direction. The higher the difference in rotation the closer

will be to 1

e.g. if the left wheel rotates backwards at speed –0.4, and the right wheel rotates forward at speed 0.5

will be 0.9

The square root gives stronger weight to smaller differences. Since component is subtracted from 1, it is maximised by robots whose wheels move in the same direction, regardless of speed and overall direction.


Last component encourages obstacle avoidance

Proximity sensors on Khepera emit a beam of infrared light – and measure quantity of reflected infrared light.

Closer a robot is to an object, the higher the measured value.

Value of i of most active sensor provides a measure of how close nearest object is.

Value subtracted from 1, so this component selects robots that stay away from objects.

Combined result of 3 components: selecting robots that move as straight as possible while avoiding obstacles in path.

control system for robot
Control system for robot

Fixed network architecture

Weights between 8 proximity sensors and 2 motor units (also bias unit)

Recurrent connections at output layer

Synaptic connections (weights) encoded as floating point numbers on chromosome

evolutionary experiments
Evolutionary experiments
  • Each generation: individuals allowed to move for 80 sensory motor loops
  • Each sensory motor loop lasted 30 ms
  • Each generation: 80 individuals
  • Evolved using roulette wheel selection, biased mutations, and one point crossover
  • After 50 generations: smooth navigation around the maze without bumping into walls.
  • Comparison between handcrafted, and evolved solution
  • Handcrafted: stopped when in looping maze (when two contralateral sensors receive the same inputs, their signals cancel each other out, and wheels don’t move)
  • Evolved solution: strong asymmetrical weights on recurrent connections to avoid deadlock situations
  • Conclusions: evolved solution is competitive to handcrafted solution.
    • Evolved solution emerges from interaction with environment
2 nd example modularity
2nd example: Modularity
  • Reasons for modularity;
    • Complexity of task
    • Improving performance
    • Design
    • Biological inspiration (and/or modelling)
  • Examples of modularity;
    • Behaviour-based robotics and subsumption architecture
  • Where do modules come from?
    • Explicit design – based on analysis/understanding of task
    • Evolving modularity: an example (Nolfi, 1997)
evolving modularity
Evolving modularity
  • Nolfi(1997) Using emergent modularity to develop control systems for mobile robots. Adaptive Behaviour 5, 3-4, 343-364
  • Usual approach to behaviour-based robotics, depends on breaking required behaviour down into subcomponents
Experiments: Khepera robot with gripper module
  • Task: removing rubbish from arena
  • Subtasks:
    • Recognise target object and orient towards it
    • Pick up target object
    • Move towards walls avoiding other target objects
    • Recognise walls, and orient so as to be able to drop object
    • Release object
Colombetti, Dorigo and Borhi (1996)
  • Studied similar task – robot collected food pieces and stored them in nest.
  • Decomposed target behaviour into simpler behaviours
    • Leave nest
    • Get food
    • Reach nest
    • Avoid obstacles
    • Coordinate behaviours
Modules trained separately and frozen
  • Coordinator module trained to achieve behaviour
  • But system depended on human division of task into behavioural modules
Nolfi (1997): used GAs to evolve NNs for control
  • Used different network architectures.
genetic algorithm
Genetic algorithm
  • For each network architecture, 100 random weight genotypes
  • Run for 15 epochs, each epoch 200 actions
  • 20 fittest reproduced (5 copies of NNs)
  • Mutations of NN: substituting 2% of randomly selected bits with new value
fitness function
Fitness function
  • How many objects released outside arena?
  • Ability to pick up targets
  • Also useful training experiences – when robot carrying target put new target in front of it.
  • 10 simulations for each of 5 architectures
  • 100 networks in each generation, run for 1000 generations
  • Number of targets released out of arena improves
Evolution: run in simulation.
  • Best controllers of generation 999 downloaded onto robots and tested in real environment for 5000 cycles
Best results from “emergent modular architecture”
  • How does network work?
  • No clear relationship between distal description of behaviour and modules
  • Each sub-behaviour is the result of the contribution of different neural modules
evolving communication for collective navigation
Evolving communication for collective navigation
  • Marocco and Nolfi (2007) Emergence of communication in embodied agents evolved for the ability to solve a collective navigation problem.
  • Collection of simulated robots, evolved to solve collective navigation problem, develop a communication system.
  • Simplified account of paper (some conditions missed out).

Robots evolved to move onto target sites in pairsDeveloped signals: a) lone robot outside target b) single robot in target area c) robot in target with another robot d) robot approaching target, interacting with robot inside.

Motor units

Com unit


Com sensors

Infrared sensors

e.g.robot in target area hearing signal c would exit

e.g. robot outside target hearing b, would approach target and signal d.

  • Use of Genetic Algorithms (GAs) in combination with neural nets
  • Introduction to GAs
    • Genotype and phenotype
    • Fitness function and idea of selective reproduction
    • Genetic operators
      • Crossover
      • Mutation
  • Example studies
    • Example of evolving a robot controller: Floreano and Mondado (1994)
    • Modular approach to behaviours: example of evolving modular decomposition (Nolfi, 1997)
    • Marocco and Nolfi (2007) evolving communication for collective navigation