socially intelligent robots n.
Skip this Video
Loading SlideShow in 5 Seconds..
Socially Intelligent Robots PowerPoint Presentation
Download Presentation
Socially Intelligent Robots

Loading in 2 Seconds...

play fullscreen
1 / 35

Socially Intelligent Robots - PowerPoint PPT Presentation

  • Uploaded on

Socially Intelligent Robots. Cynthia Breazeal MIT Media Lab Robotic Life Group. Robots turn 85 years old Posted May 31st 2006 11:38PM by Ryan Block Filed under: Robots. Dear Robots ,

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Socially Intelligent Robots

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
socially intelligent robots

Socially Intelligent Robots

Cynthia Breazeal

MIT Media Lab

Robotic Life Group


Robots turn 85 years old

Posted May 31st 2006 11:38PM by Ryan Block

Filed under: Robots

Dear Robots,

We're very sorry. It appears we missed your 85th birthday two days ago -- the anniversary of which is marked by the date Czech writer Karel Capek debuted his play R.U.R. (Rossum's Universal Robots) to its first audience in Prague. Yes, we know the concept of the automaton dates back much further, but we think it's well agreed upon that Capek's play marks the robot's entry into mass consciousness (as well as marking the first use of the word "robot"). No matter, we're just saying happy birthday, robots -- not because we fear you'll one day you'll subsume us in some dystopian nightmare of artificial intelligence gone terribly wrong, but because from Asimov to AIBO, from Roomba to Ri-Man, from QRIO to ASIMO, we just love ya. So happy birthday, happy birthday, happy birthday, robots, and when the day of reckoning comes, please remember: Engadget and its readers are your friends.

All our love,


Robots have…
    • explored ocean depths,
    • mapped subterranean mines,
    • rescued natural disaster victims,
    • assisted surgeons with operations,
    • driven autonomously across the desert,
  • And even been to Mars…
what s next
What’s Next?
  • The next big frontier…society at large
people and robots
People and Robots
  • Robots are not perceived as pure tools or appliances, but often as social actors -- over a wide range of morphologies and behaviors
robots evoke human social responses
Robots Evoke Human Social Responses

“The Kismet Effect”

New Scientist, 2005

newsmaker my friend the robot cnet news com may 24 2006
Newsmaker: My friend, the robotCNET, May 24, 2006

The PackBots have almost become members of military units, Angle said, recalling an incident when a U.S. soldier begged iRobot to repair his unit's robot, which they had dubbed Scooby Doo. "Please fix Scooby Doo because he saved my life," was the soldier's plea, Angle told the Future in Review conference last week in Coronado, Calif. For many reasons, people bond with robots in a way they don't bond with their lawn mowers, televisions or regular vacuum cleaners. At some point, this could help solve the looming health care problem caused by an enormous generation of aging people. Not only could robots make sure they take their medicine and watch for early warning signs of distress, but they could also provide a companion for lonely people and extend their independence.

social robots socio emotive factors
Social Robots Socio-emotive Factors

“Social as relationship”

Future applications

require robots to address



“Social as entertainment”

the socio-emotive and psychological

aspects of people, in long-term relations

BANDAI “elder toys”




“Social as interface”

NEC “babysitters”

OMRON “pets”

hri an emerging discipline
HRI, An Emerging Discipline

An important goal of Human-Robot Interaction (HRI) is synergy of the human-robot system. Robots bring their own abilities that complement human strengths. It is not about equivalence (replacement), but compatibility with a typical human partner

four cornerstones of social robotics in hri
Four Cornerstones of Social Robotics in HRI


Transparent Communication


Lasting Relationship

Social Learning

Social Intelligence

Cognitive Compatibility

Perspective Taking

User Studies,

Psychology &

Social Development

today s focus
Today’s Focus

Robots, like humans, should leverage the social and environmental constraints in the real world to foster learning new skills and knowledge from anyone.


Personalization agents, Adaptive user interfaces

{Lashkari, Metral, Maes, Collaborative Interface Agents, AAAI 1994}

{E. Horovitz et al., The Lumiere project, UAI 1998}

Active Learning, Learning with Queries

{Cohn, Ghahramani, Jordan, Active learning with statistical models, 1995}

{Cohn et al., Semi-supervised clustering with user feedback, 2003}

Learning by Demonstration, Programming by Example

{Voyles, Khosla, Programming robotic agents by demonstration, 1998}

{Lieberman, Your Wish is my Command, 2001}

{A. Billard, Special Issue of RAS on Robot Programming by Demonstration, 2006}

Learning by Imitation

{S. Schaal review in TICS 1999}

{K. Dautenhahn & C. Nehaniv, Imitation in Animals and Artifacts, 2002}

Animal training techniques

{Stern, Frank, Resner, Virtual Petz, Agents 1998}

{Blumberg et al. Integrated learning for interactive characters, SIGGRAPH 2002}

{Kaplan et al., Robot clicker training, RAS 2002}

Reinforcement Learning with humans

{Isbell et al. Cobot: a social reinforcement learning agent, UAI 1998}

{Evans, Varieties of Learning, AI Game Programming Wisdom, 2002}

{Clouse, Utgoff, Teaching a Reinforcement Learner, ICML 1992}

… and many more

how do ordinary people teach a rl agent
How Do Ordinary People Teach a RL Agent?

Most people don’t have experience with Machine Learning techniques, they have a lifetime of experience with social learning interactions that they bring to the table.

We emphasize the need to consider and design to support the ways that people naturally approach teaching.

And then design algorithms and systems that take better advantage of this

experiments in sophie s kitchen
Experiments inSophie’s Kitchen
  • A “computer game” - players teach a virtual robot to bake a cake, by sending various messages with a mouse interface.

Sophie learns via Q-Learning

30 steps

~10,000 states

2-7 actions/state

Allows us to run many subjects on-line

experiments in sophie s kitchen1
Experiments inSophie’s Kitchen
  • A “computer game” - players teach a virtual robot to bake a cake, by sending various messages with a mouse interface.

An object specific reward is about a particular part of the world

initial experiment
Initial Experiment
  • 18 people trained Sophie
  • They are given a description of the cake task, and told they can’t do actions but can help Sophie by sending FEEDBACK messages with the mouse
  • System logs time of state changes, agent actions, and any human feedback. We analyze games logs to understand people’s teaching behavior

Thomaz & Breazeal RO-MAN 2006

findings guidance
Findings: Guidance
  • People tried to use the object specific rewards as FUTURE directed guidance.

Never About

Most Recent Object

Always About

Most Recent Object






  • Many object rewards not about the last object used

Each player’s %Object Rewards about lastobject

Almost everyone gave rewards to the bowl or tray sitting empty on the shelf...a guidance reward.


of People

Zero rewards

to Empty Bowl

At least 1 reward

to Empty Bowl

findings people adapt teaching to their mental model of sophie
Findings: People Adapt Teaching to their Mental Model of Sophie
  • People gave more rewards after realizing their feedback made a difference
  • Interpreted Sophie’s behavior as being a “staged” learner
  • Adapted their teaching strategy accordingly







human rewards : agent actions








using guidance in sophie s kitchen
Using Guidance in Sophie’s Kitchen

Interactive Q-Learning

Algorithm, baseline system

slight delay

to animate act

and receive

human reward


gui dance experiment

Thomaz & Breazeal, AAAI 2006

  • Hypothesis: Non-expert teachers can use guidance to improve agent’s performance
  • 27 subjects trained Sophie in two groups:
    • Using feedback only
    • Using both feedback and guidance
  • Again, system logs game play and logs are analyzed to understand teaching behavior
effects of guidance
Effects of Guidance




  • 1-tailed T-tests show logs in guidance condition are significantly better than non-guidance







  • How can machine learners be Transparent?

Teachers structure the environment and the task to help a learner succeed.

Learners contribute by revealing internal state; helping the teacher maintain a mental model to make guidance more appropriate.

sophie s gaze behavior
Sophie’s Gaze Behavior

Interactive Q-Learning

Algorithm modified to

incorporate Guidance

transparency experiment
  • 52 subjects trained Sophie in an online version:
    • Feedback and guidance, no gaze
    • Feedback and guidance, Sophie gazing
  • Hypothesis:

Learners can help shape their learning environment by communicating aspects of the internal process -- gaze will improve the human’s guidance instruction

Thomaz et al., ICDL 2006

sophie s gaze behavior2
Sophie’s Gaze Behavior

Results: Sophie’s gaze significantly improves the guidance received - more when uncertainty high and less when uncertainty is low.

Uncertainty high:

3 or more choices

Uncertainty low:

3 or less

  • People bring their own teaching and learning experience to the task
    • Social factors of guidance and transparency
    • Collaborative process between teacher and learner improves performance
  • Agent can use transparency cues to improve its own learning environment by helping teacher form a better mental model
    • Adding gaze significantly improves the human’s Guidance