slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institut PowerPoint Presentation
Download Presentation
Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institut

Loading in 2 Seconds...

play fullscreen
1 / 40

Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institut - PowerPoint PPT Presentation


  • 301 Views
  • Uploaded on

The Architectural Role of Emotions in Cognitive Systems. Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute. Outline. Outline. Emotions are adaptive Can inform cognitive system design

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institut' - lotus


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

The Architectural Role of Emotions in Cognitive Systems

Jonathan Gratch

USC Institute for Creative Technologies

Joint work with

Stacy Marsella

USC Information Sciences Institute

outline
Outline

Outline

Emotions are adaptive

Can inform cognitive system design

Ground in implemented cognitive system

  • Mission Rehearsal Exercise system
  • Cognitive Appraisal Theory
  • Illustrate impact on architecture design

General implications for cognitive systems

outline3
Outline

Adaptive Role of Emotions

  • Revolutionary progress in emotion research
    • Neurophysiology of emotion (Damasio, LeDoux)
    • Appraisal theories (Frijda, Lazarus, Scherer)
  • Emotions appear adaptive (in moderation?)
    • Decision-making ─ Focus of attention
    • Learning ─ Social relationships
    • Belief formation ─ Communication
  • Growing interest in “emotional” systems

with focus on modeling human behavior

    • HCI (non-verbal recognition and generation)
    • User and human behavior modeling
    • Believability/Entertainment
outline4
Outline

Architectural Perspective

  • Can inform intelligent behavior in general
      • Motivate behavior
      • Balancing competing goals
      • Balancing reaction and deliberation
      • Disambiguating stimuli in light of existing beliefs and commitments
  • Abstract and formalize as information processing
    • Not new idea:
      • Simon(1967), Oatley&Johson-Laird(1987), Sloman
    • Revisit in light of new findings
      • In intelligent systems
      • In theories of emotion
grounding virtual humans
Grounding: Virtual Humans

Face-to-face interaction

Verbal & non-verbal behavior

Swartout,Gratch, Hill, Hovy, Johnson,

Marsella, Narayanan, Rickel, Traum, …

Marsella, Johnson & Labore

mission rehearsal exercise
Mission Rehearsal Exercise

Social Training Simulation

  • Explore high-stakes social interactions in safety of VR
mission rehearsal exercise7
Mission Rehearsal Exercise
  • Team decision-making in crisis situations:
    • Non-scripted real-time interactions
    • Planning, replanning, and plan execution
    • Teamwork, distributed authority and responsibility
    • Collaborative, mixed initiative dialogue
    • Multi-party conversations
    • Verbal and non-verbal communication
    • Emotionally-biased behavior
mission rehearsal exercise8
Mission Rehearsal Exercise
  • Assumptions/Limitations:
    • Tightly focused task-related dialogue
    • Near-expert decision makers
    • Stylized vocabulary (military speak)
    • Stylized virtual environment
slide10

Soar

Action Selection

Dialogue

Perception

Planning

Emotion

NLU pragmatics

NLG

Voice Input

World Simulator

Speech Recognition (HTK)

Semantic Parser

Projection System

Communication Bus

Animation

System

Vega

BDI

Haptek

Audio (Protools)

Motion/ Gesture Scheduler (Beat)

Speakers (10.2)

Text to Speech (Festival)

cognitive representation
Cognitive Representation

Present

Future

Past

Eagle 1-6 Assist

Desire: LT

Belief: False

Child Healthy:False

Accident

Intend: False

Blame: unresolved

Assist Eagle 1-6:False

Get Medevac

Responsibility:LT

Intend: True

Medevac Available:True

Child-Healthy

Desire: SGT

Belief: False

Probability: 75%

Soar’s Working Memory

Planning

Perception

Dialogue

Action

Soar operators

cognitive representation12
Cognitive Representation

Present

Future Plans

Past Events

Eagle 1-6 Assist

Desire: LT(+50)

Belief: False

Child Healthy:False

  • Causal Interpretation
    • Combines decision-theoretic plans with models of belief and intention
    • Uniform representation of past, present, future
    • Agent centric subjective view

Accident

Intend: False

Blame: unresolved

Assist Eagle 1-6:False

Get Medevac

Responsibility:LT

Intend: True

Medevac Available:True

Child-Healthy

Desire: SGT(+80)

Belief: False

Probability: 75%

architectural role of emotion
Architectural Role of Emotion
  • Began with view “emotion as veneer”

Ended up as central organizing construct

    • Initial problem:
      • how to convey emotion in interactive setting?
    • Built mechanism to infer plausible emotions
      • In response to simulation events
      • In response to user interventions
    • But discovered resolved architectural issues
      • Coherence is more than skin deep
      • Build it and they will come
how to convey emotion
How to convey emotion
  • Cognitive Appraisal Theory
    • Influential and well-established theory

Arnold, Frijda; Lazarus; Ortony, Clore & Collins; Scherer; Smith

    • Emphasizes tight coupling between
      • Emotion
      • Cognition
      • Motivation
cognitive appraisal theory
Cognitive Appraisal Theory

Coping

Problem-focused

Emotion-focused

Goals, Beliefs

External Events

Appraisal

Emotion

Coping

Smith and Lazarus’ cognitive-motivational-emotive system

appraisal
Appraisal

Goals, Beliefs

External Events

Appraisal

  • Appraisal = Situation assessment
    • Compare beliefs, desires and intentions
          • with
          • external circumstances
appraisal17
Appraisal
  • Characterize via appraisal variables
    • Desirability
    • Likelihood
    • Urgency
    • Unexpectedness
    • Causal attribution (causality, agency, blame/credit)
    • Coping potential (controllability, adaptability)
  • Superset of criteria considered by cog systems
    • Decision theory: desirability, likelihood
    • Scheduling: desirability, urgency
slide18

Coping Strategies

  • Coping = Response strategy
    • Characterized by ontology of coping strategies

Emotion

External Events

Goals, Beliefs

Coping

Problem-focused

Emotion-focused

slide19

Coping Strategies

  • Problem-focused (act on the world)
    • Action execution
    • Planning
    • Seek instrumental social support
  • Analogous to:
    • Deliberative or reactive problem solving
    • Team negotiation
slide20

Coping Strategies

  • Emotion-focused (act on belief)
    • Denial
    • Find silver lining
    • Shift blame
    • Distancing
  • Not typically considered by cog systems systems
    • More than a decision (e.g. abandon current plan)
      • Provides self-justification for why
      • Related to motivational / explanatory coherence
      • Leads to persistent change in behavior
modeling appraisal and coping
Modeling Appraisal and Coping

Future

Past

Soar’s Working Memory

Planning

Perception

Dialogue

Action

Soar Operators

modeling appraisal and coping22
Modeling Appraisal and Coping
  • Appraisal as plan-evaluation
    • Causal interpretation mediates agent-environment relationship
    • Define appraisal variables in terms of features of interpretation
    • Fast, reactive, parallel
  • Coping as generalized plan critics

Map to operators that change interpretation

    • Problem-focused  execute step, add plan step
    • Emotion-focused
      • Denial  Change belief
      • Find silver lining  Change utilities
      • Shift blame  Change causal attribution

 Dialogue moves

      • Distancing  Drop goal / intention
slide24

Appraisal

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame:unresolved

Child-Healthy

Desire: SGT

Satisfied: False

Perspective: Self (Sgt)

Desirability: -80

Likelihood: 100%

Blame/Credit: unresolved

Distress: 80

Sgt’s Appraisal of Accident from his perspective

slide25

Appraisal

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame: unresolved

Child-Healthy

Desire: SGT

Satisfied: False

Perspective: Lieutenant

Desirability: -80

Certainty: 100%

Blame/Credit: unresolved

Distress: 80

Distress: 80

Sgt’s Appraisal of Accident from Lieutenant’s Perspective

slide26

Coping

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame: unresolved

Child-Healthy

Desire: SGT

Satisfied: False

Distress: 80

slide27

Coping

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame: unresolved

Child-Healthy

Desire: SGT

Satisfied: False

Sgt’s Own

Perspective

Distress: 80

slide28

Coping

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame: unresolved

Get Medevac

Responsibility:LT

Child-Healthy

Desire: SGT

Satisfied: False

Probability: 75%

Make

Amends

Distress: 80

Distress: 80

Problem-Focused Coping:

Form intention to help Boy

slide29

Coping

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame: MOM

Get Medevac

Responsibility:LT

Child-Healthy

Desire: SGT

Satisfied: False

Probability: 75%

Shift

Blame

Make

Amends

Distress: 80

Distress: 80

Emotion-Focused Coping:

Blame Mother

slide30

Coping

Eagle 1-6 Assist

Desire: LT

Satisfied: False

Child Healthy:False

Assist Eagle 1-6:False

Accident

Blame: unresolved

Get Medevac

Responsibility:LT

Child-Healthy

Desire: SGT

Satisfied: False

Probability: 75%

Shift

Blame

Make

Amends

Distress: 80

Distress: 80

Personality

architectural implications
Architectural Implications
  • Emotion as central control construct
    • Planning (inform course-of-action selection)
    • NLU (inform reference resolution)
    • Dialogue (prompt dialogue initiative)
    • NLG (biases sentence generation strategies)
    • Non-verbal expression
general implications
General Implications
  • Emotion and Reflection
    • Appraisal is form of self-reflection / focus of attention
  • Emotion as decision-making
    • Generalization of decision-theory
    • More to the world than probabilities and utilities
  • Emotion and plausible reasoning
    • Emotion-focused coping motivate preference/beliefs
    • Attempt to construct coherent motivational explanation
    • Non-rational but adaptive?
general implications34
General Implications
  • Emotion and Learning
    • Focus learning on “emotionally salient” events
    • Appraisal variables as features / case indexes
conclusion
Conclusion
  • Emotion is form of information processing
    • Arguable adaptive
      • Juggling competing goals and commitments
      • Focusing cognitive resources
      • Enforcing coherence
    • Arguable unexplored by cognitive systems
slide37

Dialogue Example: Sgt’s Behavior

Secure Area

Secure 12-4

Secure 8-12

Render Aid

Secure 4-8

Secure LZ

Secure Accident

1

Focus=1

Lt: U9 “Secure a landing zone”

Committed(lt,7), 7 authorized, Obligation(sgt,U9)

Sgt: U10 “First we should secure the assembly area”

Disparaged(sgt, 7), endorsed(sgt,2)

Lt: U11“Secure the area”

Committed(lt,2), 2 authorized, Obligation(sgt,U11)

Sgt: U12 “Yes sir”

Committed(sgt,2), Push(2)

Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr})

Goal8: Start-conversation(sgt, {1sldr,2sldr,…},2)

Goal8 Sgt: U13 “Squad leaders listen up!”

Goal7 Sgt:U14 “I want 360 degree security”

Push(3)

Goal9:authorize 3

Goal9 Sgt:u15“1st squad take 12-4”

Committed(sgt,3), 3 authorized

Pop(3), Push(4)

Goal10: authorize 4

Goal10 Sgt:u16“2nd squad take 4-8”

Committed(sgt,4), 4 authorized

Pop(4)

A10: Squads move

A10: grounds U13-U18,…

ends conversation about 2, realizes 2

Pop(2), Push(7)

Decomposition

Area Secure

Squads in area

A=Lt, R=Sgt

A=Lt, R=Sgt

2

7

Decomposition

3

4

A=Sgt, R=1sldr

A=Sgt, R=2sldr

5

6

A=Sgt, R=3sldr

A=Sgt, R=4sldr

more than a theory of emotion
More than a theory of emotion
  • Appraisal as a mediating variable
    • Direct mappings (e.g. Hayes-Roth personality model)
    • Indirect mappings
      • Direct: More links, No insight on how to map
      • Indirect: more constrained. More modular

World state

Beliefs

Behavior

Desires

Personality

World state

Appraisal Variables

Beliefs

Behavior

Desires

Personality

mediating variable
Mediating Variable
  • Appraisal Mediates Personality

Personality Variable  Appraisal Variables  Behavior

e.g Extroversion  Control  Hope

Penley & Tomaka (2002)

  • Appraisal Mediates Culture

Culture Variable  Appraisal Variables  Behavior

e.g. Uncertainty avoidance  Threat  Fear

Kupperbusch et al

mediating variables
Mediating variables
  • Coping mediated by appraisal
    • Undesirable & Controllable

 Distress

 Problem directed coping

    • Undesirable & Uncontrollable

 Distress

 Emotion directed coping