Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institut

1. The Architectural Role of Emotions in Cognitive Systems Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute

2. Outline Outline Emotions are adaptive Can inform cognitive system design Ground in implemented cognitive system • Mission Rehearsal Exercise system • Cognitive Appraisal Theory • Illustrate impact on architecture design General implications for cognitive systems

3. Outline Adaptive Role of Emotions • Revolutionary progress in emotion research • Neurophysiology of emotion (Damasio, LeDoux) • Appraisal theories (Frijda, Lazarus, Scherer) • Emotions appear adaptive (in moderation?) • Decision-making ─ Focus of attention • Learning ─ Social relationships • Belief formation ─ Communication • Growing interest in “emotional” systems with focus on modeling human behavior • HCI (non-verbal recognition and generation) • User and human behavior modeling • Believability/Entertainment

4. Outline Architectural Perspective • Can inform intelligent behavior in general • Motivate behavior • Balancing competing goals • Balancing reaction and deliberation • Disambiguating stimuli in light of existing beliefs and commitments • Abstract and formalize as information processing • Not new idea: • Simon(1967), Oatley&Johson-Laird(1987), Sloman • Revisit in light of new findings • In intelligent systems • In theories of emotion

5. Grounding: Virtual Humans Face-to-face interaction Verbal & non-verbal behavior Swartout,Gratch, Hill, Hovy, Johnson, Marsella, Narayanan, Rickel, Traum, … Marsella, Johnson & Labore

6. Mission Rehearsal Exercise Social Training Simulation • Explore high-stakes social interactions in safety of VR

7. Mission Rehearsal Exercise • Team decision-making in crisis situations: • Non-scripted real-time interactions • Planning, replanning, and plan execution • Teamwork, distributed authority and responsibility • Collaborative, mixed initiative dialogue • Multi-party conversations • Verbal and non-verbal communication • Emotionally-biased behavior

8. Mission Rehearsal Exercise • Assumptions/Limitations: • Tightly focused task-related dialogue • Near-expert decision makers • Stylized vocabulary (military speak) • Stylized virtual environment

9. MRE: LeadershipTraining

10. Soar Action Selection Dialogue Perception Planning Emotion NLU pragmatics NLG Voice Input World Simulator Speech Recognition (HTK) Semantic Parser Projection System Communication Bus Animation System Vega BDI Haptek Audio (Protools) Motion/ Gesture Scheduler (Beat) Speakers (10.2) Text to Speech (Festival)

11. Cognitive Representation Present Future Past Eagle 1-6 Assist Desire: LT Belief: False Child Healthy:False Accident Intend: False Blame: unresolved Assist Eagle 1-6:False Get Medevac Responsibility:LT Intend: True Medevac Available:True Child-Healthy Desire: SGT Belief: False Probability: 75% Soar’s Working Memory Planning Perception Dialogue Action Soar operators

12. Cognitive Representation Present Future Plans Past Events Eagle 1-6 Assist Desire: LT(+50) Belief: False Child Healthy:False • Causal Interpretation • Combines decision-theoretic plans with models of belief and intention • Uniform representation of past, present, future • Agent centric subjective view Accident Intend: False Blame: unresolved Assist Eagle 1-6:False Get Medevac Responsibility:LT Intend: True Medevac Available:True Child-Healthy Desire: SGT(+80) Belief: False Probability: 75%

13. Architectural Role of Emotion • Began with view “emotion as veneer” Ended up as central organizing construct • Initial problem: • how to convey emotion in interactive setting? • Built mechanism to infer plausible emotions • In response to simulation events • In response to user interventions • But discovered resolved architectural issues • Coherence is more than skin deep • Build it and they will come

14. How to convey emotion • Cognitive Appraisal Theory • Influential and well-established theory Arnold, Frijda; Lazarus; Ortony, Clore & Collins; Scherer; Smith • Emphasizes tight coupling between • Emotion • Cognition • Motivation

15. Cognitive Appraisal Theory Coping Problem-focused Emotion-focused Goals, Beliefs External Events Appraisal Emotion Coping Smith and Lazarus’ cognitive-motivational-emotive system

16. Appraisal Goals, Beliefs External Events Appraisal • Appraisal = Situation assessment • Compare beliefs, desires and intentions • with • external circumstances

17. Appraisal • Characterize via appraisal variables • Desirability • Likelihood • Urgency • Unexpectedness • Causal attribution (causality, agency, blame/credit) • Coping potential (controllability, adaptability) • Superset of criteria considered by cog systems • Decision theory: desirability, likelihood • Scheduling: desirability, urgency

18. Coping Strategies • Coping = Response strategy • Characterized by ontology of coping strategies Emotion External Events Goals, Beliefs Coping Problem-focused Emotion-focused

19. Coping Strategies • Problem-focused (act on the world) • Action execution • Planning • Seek instrumental social support • Analogous to: • Deliberative or reactive problem solving • Team negotiation

20. Coping Strategies • Emotion-focused (act on belief) • Denial • Find silver lining • Shift blame • Distancing • Not typically considered by cog systems systems • More than a decision (e.g. abandon current plan) • Provides self-justification for why • Related to motivational / explanatory coherence • Leads to persistent change in behavior

21. Modeling Appraisal and Coping Future Past Soar’s Working Memory Planning Perception Dialogue Action Soar Operators

22. Modeling Appraisal and Coping • Appraisal as plan-evaluation • Causal interpretation mediates agent-environment relationship • Define appraisal variables in terms of features of interpretation • Fast, reactive, parallel • Coping as generalized plan critics Map to operators that change interpretation • Problem-focused  execute step, add plan step • Emotion-focused • Denial  Change belief • Find silver lining  Change utilities • Shift blame  Change causal attribution  Dialogue moves • Distancing  Drop goal / intention

23. Émile: Architectural Manifestation The Emotional Octopus

24. Appraisal Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame:unresolved Child-Healthy Desire: SGT Satisfied: False Perspective: Self (Sgt) Desirability: -80 Likelihood: 100% Blame/Credit: unresolved Distress: 80 Sgt’s Appraisal of Accident from his perspective

25. Appraisal Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Child-Healthy Desire: SGT Satisfied: False Perspective: Lieutenant Desirability: -80 Certainty: 100% Blame/Credit: unresolved Distress: 80 Distress: 80 Sgt’s Appraisal of Accident from Lieutenant’s Perspective

26. Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Child-Healthy Desire: SGT Satisfied: False Distress: 80

27. Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Child-Healthy Desire: SGT Satisfied: False Sgt’s Own Perspective Distress: 80

28. Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Get Medevac Responsibility:LT Child-Healthy Desire: SGT Satisfied: False Probability: 75% Make Amends Distress: 80 Distress: 80 Problem-Focused Coping: Form intention to help Boy

29. Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: MOM Get Medevac Responsibility:LT Child-Healthy Desire: SGT Satisfied: False Probability: 75% Shift Blame Make Amends Distress: 80 Distress: 80 Emotion-Focused Coping: Blame Mother

30. Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Get Medevac Responsibility:LT Child-Healthy Desire: SGT Satisfied: False Probability: 75% Shift Blame Make Amends Distress: 80 Distress: 80 Personality

31. MRE: LeadershipTraining

32. Architectural Implications • Emotion as central control construct • Planning (inform course-of-action selection) • NLU (inform reference resolution) • Dialogue (prompt dialogue initiative) • NLG (biases sentence generation strategies) • Non-verbal expression

33. General Implications • Emotion and Reflection • Appraisal is form of self-reflection / focus of attention • Emotion as decision-making • Generalization of decision-theory • More to the world than probabilities and utilities • Emotion and plausible reasoning • Emotion-focused coping motivate preference/beliefs • Attempt to construct coherent motivational explanation • Non-rational but adaptive?

34. General Implications • Emotion and Learning • Focus learning on “emotionally salient” events • Appraisal variables as features / case indexes

35. Conclusion • Emotion is form of information processing • Arguable adaptive • Juggling competing goals and commitments • Focusing cognitive resources • Enforcing coherence • Arguable unexplored by cognitive systems

37. Dialogue Example: Sgt’s Behavior Secure Area Secure 12-4 Secure 8-12 Render Aid Secure 4-8 Secure LZ Secure Accident 1 Focus=1 Lt: U9 “Secure a landing zone” Committed(lt,7), 7 authorized, Obligation(sgt,U9) Sgt: U10 “First we should secure the assembly area” Disparaged(sgt, 7), endorsed(sgt,2) Lt: U11“Secure the area” Committed(lt,2), 2 authorized, Obligation(sgt,U11) Sgt: U12 “Yes sir” Committed(sgt,2), Push(2) Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr}) Goal8: Start-conversation(sgt, {1sldr,2sldr,…},2) Goal8 Sgt: U13 “Squad leaders listen up!” Goal7 Sgt:U14 “I want 360 degree security” Push(3) Goal9:authorize 3 Goal9 Sgt:u15“1st squad take 12-4” Committed(sgt,3), 3 authorized Pop(3), Push(4) Goal10: authorize 4 Goal10 Sgt:u16“2nd squad take 4-8” Committed(sgt,4), 4 authorized Pop(4) … A10: Squads move A10: grounds U13-U18,… ends conversation about 2, realizes 2 Pop(2), Push(7) Decomposition Area Secure Squads in area A=Lt, R=Sgt A=Lt, R=Sgt 2 7 Decomposition 3 4 A=Sgt, R=1sldr A=Sgt, R=2sldr 5 6 A=Sgt, R=3sldr A=Sgt, R=4sldr

38. More than a theory of emotion • Appraisal as a mediating variable • Direct mappings (e.g. Hayes-Roth personality model) • Indirect mappings • Direct: More links, No insight on how to map • Indirect: more constrained. More modular World state Beliefs Behavior Desires Personality World state Appraisal Variables Beliefs Behavior Desires Personality

39. Mediating Variable • Appraisal Mediates Personality Personality Variable  Appraisal Variables  Behavior e.g Extroversion  Control  Hope Penley & Tomaka (2002) • Appraisal Mediates Culture Culture Variable  Appraisal Variables  Behavior e.g. Uncertainty avoidance  Threat  Fear Kupperbusch et al

40. Mediating variables • Coping mediated by appraisal • Undesirable & Controllable  Distress  Problem directed coping • Undesirable & Uncontrollable  Distress  Emotion directed coping