Functional Contributions of Emotion to Artificial Intelligence

1. Functional Contributions of Emotion to Artificial Intelligence Bob Marinier Advisor: John Laird

2. Introduction • Folk psychology considers emotions to be a distraction from logical thought • People tend to think that emotion is unknowable, indefinable • Psychological work in the last several decades has demonstrated that emotion plays a critical role in effective functioning and learning

3. Introduction • Research Goals • Bring the functionality of emotion to AI • Create a precise computational definition of emotion • Approach • Integrate emotion with a complete agent framework • Computationally distinguish emotion, mood and feeling • Weight feeling’s importance by computing its intensity • Use feeling as intrinsic reward signal to drive reinforcement learning

4. Appraisal Theories of Emotion • A situation is evaluated along a number of appraisal dimensions,many of which relate the situation to current goals • Novelty, goal relevance, goal conduciveness, expectedness, causal agency, etc. • Result of appraisals determines emotion • The emotion is combined with mood, which is an “average” over recent emotions, to form a feeling, which is actually perceived with some intensity • The feeling can then be coped with (via internal or external actions) Situation Goals Coping Appraisal Emotion, Mood, Feeling

5. Appraisals to Emotions(Scherer 2001) • Why these dimensions? • What is the functional purpose?

6. Functions of Emotion • Situation summary: Appraisals and emotion provide abstract interpretation • Decouples stimulus/response: Can react to interpretation instead of stimulus • Attention: Some appraisals help prioritize processing • Historical context: Mood provides a context for current interpretations • Learning: Feeling may provide an intrinsic reward signal • Memory • Decision making • Action preparation • Communication

7. Outline • Integrate emotion with a complete agent framework • Computationally distinguish emotion, mood and feeling • Weight feeling’s importance by computing its intensity • Use feeling as intrinsic reward signal to drive reinforcement learning • Discussion & Conclusion Situation Goals Coping Appraisal Emotion, Mood, Feeling

8. Newell’s Abstract Functional Operations(Newell 1990) • Allen Newell defined a set of computational Abstract Functional Operations that are necessary and sufficient for immediate behavior in humans and complete agents

9. Newell’s Abstract Functional Operations(Newell 1990) • …but how these actually work was not clear.

10. NAFO and Appraisal(Marinier & Laird 2006)

11. Outline • Integrate emotion with a complete agent framework • Computationally distinguish emotion, mood and feeling • Weight feeling’s importance by computing its intensity • Use feeling as intrinsic reward signal to drive reinforcement learning • Discussion & Conclusion Situation Goals Coping Appraisal Emotion, Mood, Feeling

12. Extending Soar with Emotion(Marinier & Laird 2007) • Soar is a cognitive architecture • A cognitive architecture is a set of task-independent mechanisms that interact to give rise to behavior • Cognitive architectures are general agent frameworks Episodic Semantic Symbolic Long-Term Memories Procedural Semantic Learning Episodic Learning Chunking Reinforcement Learning Feeling Generation Short-Term Memory Situation, Goals Decision Procedure Visual Imagery Perception Action Body

13. Extending Soar with Emotion(Marinier & Laird 2007) Episodic Semantic Symbolic Long-Term Memories Procedural Semantic Learning Episodic Learning Chunking Reinforcement Learning +/- Intensity Feeling Generation Feeling .9,.6,.5,-.1,.8,… Short-Term Memory Situation, Goals Feelings Decision Procedure Feelings Appraisals Visual Imagery Emotion .5,.7,0,-.4,.3,… Mood .7,-.2,.8,.3,.6,… Perception Action Knowledge Body Architecture

14. Computing Feeling from Emotion and Mood(Marinier & Laird 2007) • Assumption: Appraisal dimensions are independent • Limited Range: Inputs and outputs are in [0,1] or [-1,1] • Distinguishability: Very different inputs should lead to very different outputs • Non-linear: Linearity would violate limited range and distinguishability

15. Example

16. Maze Task Start Goal

17. Feeling Dynamics Results very easy

18. Computing Feeling Intensity(Marinier & Laird 2007) • Motivation: Intensity gives a summary of how important (i.e., how good or bad) the situation is • Limited range: Should map onto [0,1] • No dominant appraisal: No single value should drown out all the others • Can’t just multiply values, because if any are 0, then intensity is 0 • Realization principle: Expected events should be less intense than unexpected events

19. Example

20. Outline • Integrate emotion with a complete agent framework • Computationally distinguish emotion, mood and feeling • Weight feeling’s importance by computing its intensity • Use feeling as intrinsic reward signal to drive reinforcement learning • Discussion & Conclusion

21. Intrinsically MotivatedReinforcement Learning(Sutton & Barto 1998; Singh et al. 2004) External Environment Environment Actions Sensations Critic “Organism” Internal Environment Actions Rewards States Critic Appraisal Process Agent +/-Feeling Intensity Decisions Rewards States Agent

22. Learning Task Start Goal

23. Learning Results

24. Discussion & Conclusion • Discussion • Agent learns fast • Gets frequent reward signals • Mood accelerates learning • Provides reward during those steps in which the agent has no emotion • Conclusion • Developed an initial computational model of emotion • Integrated model with complete agent framework • Demonstrated some functional advantages of integration