Michal de Vries Human-robot interaction
Breazeal, Brooks, Gray, Hoffman, Kidd, Lee, Lieberman, Lockerd and Mulanda International Journal of Humanoid Robots Submitted 2003, published in 2004 Humanoid robots as cooperative partners for people
Overview • 1) Introduction • 2) Understanding others • 3) Social robotics • 4) Meet Leonardo • 5) Task learning • 6) Discussion
1) Introduction • The goal of the paper: developing robots with social abilities. • Such robots can understand natural human instructions (such as natural language, gestures, emotional expressions). • Learning new skills should be done quickly. • Create robots that play a role in the daily lives of ordinary people.
2) Understanding others • Theory of Mind: People attribute mental states (beliefs, desires, goals) to understand and predict behavior. • This is even the case with non-living things of sufficient complexity (Breitenberg). • Although this is far from scientific, it is suprisingly useful (Dennett, 1987). • Mirror neurons are a possible neural mechanism for the Theory of Mind (Gallese & Goldman, 1998).
3) Social robots • Human-robot collaboration: No master-slave relation between human and robot, but cooperating partners. • Joint Intention Theory: Doing something together as a team where the teammates share the same goal and same plan of execution.
Robots understanding others • Most robots interact with people as objects or socially impaired people. • Social robots must be capable of understanding the intentions, beliefs, goals and desires of people. It must also understand social cues (and vice versa). • Such robots must be able to take multiple points of view. Common vs. partial knowledge.
How should social robots learn? • It is a trend in machine learning to eschew built-in structure or a priori knowledge of the environment. The main focus is on statistical learning techniques. • Such techniques need hunderds or thousands examples to learn something succesfully.
How should robots learn? • Learning without built-in structure is a problem: A robot needs to learn quickly. Learning in biology is robust and fast. • Furthermore, humans are also born with innate cognitive and behavioral machinery which develops in an environment. • So the authors use a combination of bottom-up and top-down processing.
Leonardo: a robot with 65 degree of freedom 4) Meet Leonardo
Leo's computational architecture Leonardo's architecture
Understanding speech • Leo cannot speak, but has a natural language understanding system called Nautilus. • Nautilus supports: For instance, a basic vocabulary, simple contexts and spatial relations.
The vision system • Leo percieves the environment with 3 camera systems. • A camera behind the robot to track people and objects in Leo's environment (peripheral information). • An overhead camera mounted in the ceiling that faces vertically down to track gestures and objects (color, position, shape, size). • The third camera system is in Leo's eyes and processes face recognition and facial features.
Attention • Leo's attentional system computes the level of saliency (interest) for objects and events. • Three factors compute saliency: perceptual properties internal states (belief system) socially directed reference
Beliefs • Seeing reflects the state of the world as it is directly precieve. • Beliefs are representational and are held even if they do not happen to agree with immediate perceptual experience. • Leo's belief system gets input (visual, tactile information and speech) and merges this information into a coherent set of beliefs.
Beliefs • Beliefs must be processed and updated correctly. • Leo can compare his beliefs with beliefs of others. It must make a distinction between his beliefs and the beliefs of others, but also know which beliefs are common knowledge. • Leo can represent beliefs of others by monitoring people's looks at objects, their gestures and talks over time.
5) Task learning • Leo can learn from natural human instructions. • “Leo, this is a hammer.” • By hearing its own name, the word “this” and in combination with a pointing gesture to a hammer, the speech understanding passes this knowledge to the spatial reasoning and belief system. • Leo can show whether he understands the instructions. “Leo, show me the hammer.” • Leo can evaluate its own capabilities.
Push the button Leo learns pushing buttons
How Leo learns pushing buttons • Task:“Buttons-On-and-Off” Leo indicates that he does not know such task and goes into learning mode. • Subtask:“Button-On” The same reaction. • A person learns Leo this by demonstrating it. It says “Press button 1” and turns it on. • Leo encodes the goal state associated with an action performed in the tutorial setting by comparing the world state before and after its execution. • The same holds for the subtask “Button-Off”.
A schematic overview All subtasks must be learned in order to master the overall task.
6) Discussion • Social skills (performing/recognizing) are important for robots in relation with humans. • No master-slave relation, but collaboration. • Knowing what matters: attention. • An action can be determined in more ways: Reinforcement learning (large state-action spaces -> large number of trials) Learning by imitation (a lot faster, but requires innate knowlegde)
Some remarks • Leo cannot speak, but speech is very important in social interaction. • The practical and biological implausibility of 3 camera systems, especially an overhead camera. • The biological implausibility of innate knowledge. Of course, we are biased towards some behaviour, but we do not have an a priori vocabulary. • No role for mirror neurons?
Extra Info • More information about Leo can be found at:http://robotic.media.mit.edu/projects/robots/leonardo/overview/overview.html • Questions?