Modeling the development of mirror neurons

1. search rotation Modeling the development of mirror neurons identify and localize object Giorgio Metta, Paul Fitzpatrick • MIT AI Lab, Humanoid Robotics Group • {pasa,paulfitz}@ai.mit.edu Problem Question: How do mirror neurons develop? Answer: Two stage hypothesis: learn about objects and then imitate the goal of an action if directed towards the same object. Motivation Our inspiration comes from neuroscience: in particular, mirror neurons. This is a class of neurons found in the monkey's frontal cortex (area F5). A particular mirror neuron is activated both when the monkey executes an action and when it observes the same action performed by somebody else. Mirror neurons can be related to gesture recognition, language, and learning by imitation. Previously-poked prototypes Solution Our solution We used poking and prodding as a precursor to full-blown manipulation 1. Learn about object affordances (canonical neurons) 2. Interpret observations on the basis of affordances (mirror) 3. Poke/push objects according to their affordances 4. Mimicry 1 2 Facts F5 contains two classes of neurons: canonical and mirror. Both classes respond when object-directed actions are executed. Canonical neurons respond to the presentation of an object according to the potential grasp types the object affords. Mirror neurons respond to an observed action in accordance to the grasp type. 3 4 Object, goal connects robot and human action 1 2 side tap back slap 0.5 0.25 0.35 pull in Bottle, “pointiness”=0.13 0.45 0.3 0.2 0.4 0.25 0.35 0.15 0.2 0.3 P(x) P(x) 0.15 estimated probability of occurrence 0.25 0.1 side tap 0.2 0.1 0.05 Rolls at right angles to principal axis 0.15 0.05 0.1 0 0 -200 -150 -100 -50 0 50 100 150 200 -200 -150 -100 -50 0 50 100 150 200 0.05 direction of movement [degrees] direction of movement [degrees] 0 0 10 20 30 40 50 60 70 80 90 difference between angle of motion and principal axis of object [degrees] push away Initial position back slap Clustering Motor vocabulary Learn about object affordances The affordance considered is the direction of movement with respect to the object’s principal axis Final position A toy car: it rolls in the direction of its principal axis A bottle: it rolls orthogonal to the direction of its principal axis Interpreting observations A foreign manipulator (human) pokes an object The direction of movement is compared with the object afforadance A toy cube: it doesn’t roll, it doesn’t have a principal axis A ball: it rolls, it doesn’t have a principal axis side tap Learn the “geometry” of poking E.g. poking from the left causes the object to move to the right back slap Initial position Final position 3 Exploiting affordances during poking When a known object is presented to the robot, it chooses the appropriate action to make the object roll Example 1 Initial position Final position Example 2 Mimicry The robot mimics the observed action trying to fulfill the goal rather than an actual movement 4 Acknowledgments: This work is funded by DARPA as part of the “Natural Tasking of Robots Based on Human Interaction Cues” project under contract number DABT 63-00-C-10102.