Interface Agents

1. Interface Agents Von-Wun Soo Department of Computer Science National Tsing Hua University

2. Importance of intelligent interface agents • Education and tutorial agents • Entertainment agents • Cartoon/commercial characters: • singers • news reporters • User friendly interface

3. Outline • Introduction • Animated Agents • Dialogue Agents • Emotional Models

4. Issues in Interface Agents • Animation and humanoid behavior • Facial expression • Multi-modal interaction • Gesture, facial expression, text, speech, graph, video • Natural language interface • Speech and dialogue management • Natural texts • Model of emotional states • Artificial life– vivid imitation of fish, bird, etc.

5. Projects on Animated Agents • Peedy, Conversational assistant (Microsoft Research) • Multimodal conversation with social agents (Sony computer science Lab) • Jack, articulated figure with realistic behavior (HMS, university of Pennsylvania) • The Woggles, individual personalities which engage in social behaviors and display emotions (Oz Project, CMU)

6. Jack • Jack is a software package developed at the Center for Human Modeling and Simulation at the University of Pennsylvania, • and is available from UGS. • Jack provides a 3D interactive environment for controlling articulated figures. • It features a detailed human model and includes • realistic behavioral controls, • anthropometric scaling, • task animation and evaluation systems, • view analysis, • automatic reach and grasp, • collision detection and avoidance,

7. Steve-- Pedagogical agent • (Soar Training Expert for Virtual Environments) that supports the learning process. • Steve agents can demonstrate skills to students, answer student questions, watch the students as they perform the tasks, and give advice if the students run into difficulties. • Multiple Steve agents can inhabit a virtual environment, along with multiple students. This helps make it possible to train students on team tasks.

8. Peedy

9. Projects on Animated Interface Agents • Herman, the Pedagogical Agents of Design-A-Plant (North Carolina State University) • Persona, the animated Presentation Agent of the PPP system (DFKI GmbH) • Cosmo, the Pedagogical Agent of the Internet Advisor System (North Carolina State University) • Info Wiz: An Animated Voice Interactive Interface to the Web (SRI International)

10. Extempo Agents • Extempo: http://www.extempo.com/

11. Microsoft’s PERSONA • PERSONA 計畫目標為1. 具 擬人化 anthromophoism 介面 or life-like characters 2自然語言對話溝通 3. 個人化服務 . • PERSONA 內的三個元件: • 1. Animation • 2. NLP processing : syntax, semantic, dialogue • 3. Text to speech • Peedy: http://msdn.microsoft.com/workshop/imedia/agent/TryMicrosoftAgent.htm

12. MIT AVATAR project • (http://avatars.www.media.mit.edu/avatars/

13. MIT medium lab projects l REA project: Mimic human real-estate agent to interact with users in natural language. • Gandalf project: Solar system experts to talk with humans in natural language and gestures about traveling in planets http://gn.www.media.mit.edu/groups/gn/index.html

14. Discourse research • Kathaleen McCoy http://www.eecis.udel.edu/~mccoy/, • Sandra Caberry http://www.eecis.udel.edu/~carberry/

15. SRI International’s CommandTalk • Allow commander in chief to control battle field in simulation using natural language • http://www.ai.sri.com/~lesaf/brochure.html

16. Theories and Models of Emotion • OCC theory of motions: • Ortony A. et al. The cognitive structure of emotions, Cambridge University Press,1988 • Fidja, Emotions, Cambridge University Press, 1987.

17. OCC model • 22 emotion categories are modeled according to: • Event consequence with respect to goal or desire toward self or other agents • (Happy-for, gloating, resentment, pity) (joy, distress) (future prospect: satisfaction, relief, fear-confirm, dissapointment) • Agent: actions from self or other agent with respect to standards and praise-worthiness • (pride, shame, admiration, reproach) • (anger, remorse, gratification, gratitude) • Object: attitude toward objects (like or dislike) (love, hate)

18. Projects with emotion agents • Em project: CMU by W. Reily • CyberCafe, Virtual Theater: Stanford, KSL, by B. Hayes-Roth • Cosmo and Herman: by Lester • ALIVE: MIT by P. Maes

19. Extempo Agents • Extempo: http://www.extempo.com/

20. Microsoft’s PERSONA • PERSONA 計畫目標為1. 具 擬人化 anthromophoism 介面 or life-like characters 2自然語言對話溝通 3. 個人化服務 . • PERSONA 內的三個元件: • 1. Animation • 2. NLP processing : syntax, semantic, dialogue • 3. Text to speech • Peedy: http://msdn.microsoft.com/workshop/imedia/agent/TryMicrosoftAgent.htm

21. MIT AVATAR project • (http://avatars.www.media.mit.edu/avatars/

22. MIT medium lab projects l REA project: Mimic human real-estate agent to interact with users in natural language. • Gandalf project: Solar system experts to talk with humans in natural language and gestures about traveling in planets http://gn.www.media.mit.edu/groups/gn/index.html

23. Discourse research • Kathaleen McCoy http://www.eecis.udel.edu/~mccoy/, • Sandra Caberry http://www.eecis.udel.edu/~carberry/

24. SRI International’s CommandTalk • Allow commander in chief to control battle field in simulation using natural language • http://www.ai.sri.com/~lesaf/brochure.html

25. Theories and Models of Emotion • OCC theory of motions: • Ortony A. et al. The cognitive structure of emotions, Cambridge University Press,1988 • Fidja, Emotions, Cambridge University Press, 1987.

26. OCC model • 22 emotion categories are modeled according to: • Event consequence with respect to goal or desire toward self or other agents • (Happy-for, gloating, resentment, pity) (joy, distress) (future prospect: satisfaction, relief, fear-confirm, dissapointment) • Agent: actions from self or other agent with respect to standards and praise-worthiness • (pride, shame, admiration, reproach) • (anger, remorse, gratification, gratitude) • Object: attitude toward objects (like or dislike) (love, hate)

27. Projects with emotion agents • Em project: CMU by W. Reily • CyberCafe, Virtual Theater: Stanford, KSL, by B. Hayes-Roth • Cosmo and Herman: by Lester • ALIVE: MIT by P. Maes

28. Virtual Character and Interactive Drama

29. Problems and Motivations • Role playing and games • Education in Entertainment (寓教於樂)。 • Can a virtual character believable?acting? • Virtual character must • have enough knowledge to react properly in the game • have proper emotion responses • Social cognition and reasoning • Interacting games must has proper development of story to increase interestingness • Players must has some controlling power to the development of the game

30. Emotional Models • To provide an appraisal model to evaluate a specific event, action, and object with respect to the response of emotions toward the social goals, norms andattitudes of an agent • For example: • Failure (success) in a game will cause one to generate the emotional response of disappointment (excitement) or discourage (happy) • Generate the emotion of disgust, angry, contemplateness while witness someone violating traffic rules

31. OCC Emotional model • Ortony, A., Clore, G. L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press (1988) • 22 emotion types are modeled and are classified into 3 categories according to events, actions, and objects • + surprise and disgust (24 emotions) • 6 expression groups: joy, sadness, anger, fear, surprise, disgust)

32. OCC Model • Provides an appraisal model to evaluate particular events; actions; or objects perceived from environment wrt goals; norms; attitudes of the agents and their societies. • The appraisal model provides “reasonable” emotional response for those events, actions and objects.

33. OCCModel

34. Emotions of WELLBEING • joy and distress • characterize positive and negative changes about one’s own utility, the status of well-being. • The factors are: desirability of self, undesirability of self, desirability of others, undesirability of others, pleasingness, and displeasingness. • pleasingness may directly come from agent’s utility value, • the desirability and undesirability depend on both stimuli from self and others. • With this extended model, an agent may feel joyful because others inform it that its current status is pretty well.

35. Emotions of ATTRIBUTION • pride, shame, admiration, and reproach. • when the agent appraises at the result of an action set by either its own or others’. • Emotional elicitation factors for ATTRIBUTION: • praiseworthiness of self, blameworthiness of self, praiseworthiness of others, and blameworthiness of others. • Emotions toward others would be further influenced by agent’s social relations. • The agent would tend to admire more about its friend’s praiseworthy action, and to reproach more for its enemy’s blameworthy action.

36. Emotions of ATTRACTION • appealingness and disappealingness to objects. • appealingness controls love and disappealingness controls hate. • The weight of links in this component can be seen as how easy of an agent would like something. • Agents might also develop both love and hate emotions at the same time based on the different characteristics of the same object.

37. Emotions of COMPOUNDS • gratification, remorse, gratitude, and anger. • compounds of WELLBEING and ATTRIBUTION. • When a consequence of an event is caused by an action, the compound emotions will be elicited. • For example, if an agent is attacked by its enemy, it may feel unhappy about getting hurt, reproach to its enemy’s attack and feel angry about this situation. • The intensities of emotions in this component are influenced by the intensities of emotions of WELLBEING and ATTRIBUTION.

38. Emotion and Social Constraints on a Virtual Character • The behaviors of agents are modified by emotions and social effects • The agents might behave “irrationally” e.g. not necessary “good” to themselves. • The agents might behave (express) differently due to different emotional modes or states. (facial expressions; speech tones; gestures, etc.) • The agents might behave differently due to different social relations to other agents, e.g. enemies or friends in response to the same events

39. Emotion and Social reasoning • Relations of enemy and friends can have different emotional responses • Example: • To see a good friend in car accident will generate sympathy • To see an enemy in a car accident to generate the emotion of gloating 幸災樂禍

40. Moods • Reflect prolonged state of mind, Cumulated states of emotion • A mood is a relatively lasting affective state. Moods differ from emotions in that they are less specific, often less intense, less likely to be triggered by a particular stimulus or event, however longer lasting.[1

41. Moods • Moods generally have either a positive or negative valence. In other words, people often speak of being in agood or badmood. • moods generally last for hours or days • Mood also differs from temperament or personality traits which are even more general and long lasting.

42. Personality model • Ocean Model ofPersonality: • 5 factors: Extraversion (Intraversion), 外向Agreeableness 親切, Conscientiousness (Lack of conscientiousness) 良心信心, Neuroticism (Emotional Stability)神精質 and Openness(closeness to experiences)開明,心胸寬大 • Other model of personality: • Optimism 樂觀; pessimism 悲觀

43. 6 facets (or subordinate traits).

44. Criticism of Big-five factor model • Block (1995) gave a detailed critique of the Big Five in A contrarian view of the five-factor approach to personality description. Costa and McCrae (1995) answered this paper in Solid ground in the wetlands of personality: A reply to Block.

45. One common criticism is that the Big Five does not explain all of human personality. • It ignores such as Religiosity, Manipulativeness/ Machiavellianism, Honesty, Thriftiness, Conservativeness, Masculinity/Femininity, Snobbishness, Sense of humour, Identity, Self-concept, and Motivation.

46. Relation between OCEAN and OCCmodels

47. The case of interaction between Personality and Emotion • Julie is standing outside. She has to carry a heavy box upstairs. • A passingman offers to help her carry the box upstairs. • Julies personality has a big influenceon her perception and on her behaviour. • If she has an extravert personality,she will be happy that someone offers her some help. • If she has a highly introvertand/or neurotic personality, she will feel fear and distress and she will responddifferently.

49. Rules of generating mixed emotions

50. Mixing emotions of WELLBEING and ATTRIBUTION for self