Emotion in Interaction: Embodied Conversational Agents

1. Emotion in Interaction: Embodied Conversational Agents IUT de Montreuil Université Paris 8 Catherine Pelachaud

2. Emotion in Interaction: Humaine • Three domains of study • Perception domain: how certain aspects related to perception may influence agent’s actions • Interaction domain: how to create relations between users and agents; how the agent can provide feedback • Generation domain: how to show expressive behaviours consistently and naturally across modalities

3. Perception and Attention • Non obtrusive agent: based on the interest shown by the addressee decides • when to start a communication • how long to maintain it • when to finish it • Perception of attentive and emotional behaviors • where to look at • how to interpret other’s behaviors

4. Perception and Attention • Endow an ECA with the capabilities to: • have a visual attention and perception model • show attention and interest • attract the attention of others

5. Interaction • Interaction: two or more parties exchange messages. • Interaction is by no means a one way communication channel between parties. • Within an interaction, parties take turns in playing the roles of the speaker and of the addressee.

6. Interaction • Speaker and addressee adapt their behaviors to each other • Speaker monitors addressee’s attention and interest in what he has to say • Addressee selects feedback behaviors to show the speaker that he is paying attention

7. Interaction • Agent as adressee ought to have the capabilities: • attention: pay attention to the signals produced by speaker to perceive, process and memorize them • perception: of signals • comprehension: understand meaning attached to signals • internal reaction: the comprehension of the meaning may create cognitive and emotional reaction • decision: decision to communication the internal reaction • generation: display behaviors

8. Interaction • Providing feedback is a fundamental aspect of engagement • Backchannel: signals of • 1-perception • 2-attention -- interest • 3-understanding • 4-attitude: belief (T/F); like (T/F); • 5- acceptance • positive/negative • any combination of the above • pay attention but not understand; understand but not believe

9. Expressive Behaviors • Behaviors encode: • content information (the ‘What is communicating’) • expressive information (the ‘How it is communicating’) • highly synchronized • multimodal

10. Expressive Behaviors • Basic emotions • Anger, fear, sadness, happiness, disgust, surprise • Universally recognized (Ekman) • Basic emotion = family of related states (Ekman 75) • Real life emotions are often complex and involve several emotions

11. Expressive Behaviors • Blends may occur either as • superposition of emotions • masking of one emotion by another emotion • quick succession of different emotions • suppression / overacting of an emotion • Superposition of 2 emotions: evaluation of an event can happen under different angles • Mask an emotion by another one : consideration of social context

12. Examples • Disappointment Masked by Joy • Superposition of Sadness and Joy Disappointment joy Disappointment masked by joy Sadness Joy Sadness and Joy

13. Callas • Creation of a virtual agent able to • perceive and be attentive • be aware of and adapt to the surroundings (physical and social context) • interact with users/other agents • communicate and listen • show ‘real’ emotions • use expressive gestures

14. Callas • Extension and refinement of the results obtained in Humaine • Real-Time integrated system • Integration of virtual and real perception models • Real situations