Adapting OCC theory for affect perception in educational software

1. Adapting OCC theory for affect perception in educational software George Katsionis, Maria Virvou Department of Informatics University of Piraeus Piraeus 18534, Greece gkatsion@kman.gr;mvirvou@unipi.gr

2. Educational Software • Due to the potential benefits of computer assisted learning there is a growing interest for educational software from numerous institutions. • Educational applications have to incorporate as many reasoning abilities as possible to be educationally beneficial. • In order to do that, such software has to be structured in such a way as to be highly adaptive and individualised to the needs of each student.

3. The role of affect in learning • How people feel plays an important role on their cognitive processes as well. • Researchers seeking to create intelligent applications begin to realise the importance of emotions in attention, planning, learning, memory, and decision-making. • There has been an increasing interest in building emotionally intelligent interactive systems that are equipped to express emotionsor even understand emotions.

4. Emotionally intelligent systems • Implement an emotion model which is responsible for emotion synthesis (Roseman, Antoniou & Jose 1996; Sloman 1999; Ortony, Clore & Collins (1988). • Most of the research about implementing an emotion theory concerns the use of emotions for making more lifelike, believable and pedagogical agents for interactive systems. • There are some studies that are focusing on detecting emotions of the users, but are mainly referring to restricted environments and specific conditions.

5. OCC Cognitive Theory of Emotions • The OCC (Ortony, Clore, & Collins, 1988) model has established itself as the standard model for emotion synthesis. • A large number of studies employed the OCC model to generate emotions for their embodied characters, and lately there are some studies, like our case, that are using it to model user emotional states.

6. OCC Cognitive Theory of Emotions This model specifies 22 emotion categories based on: • Reactions of the subject to an event which is relevant to his/hers goals or someone else goals depending on the desirability of the event. • Reactions of the subject to an action of an accountable agent (including itself) depending on the praiseworthiness of the action. • Reactions of the subject to attitudes of attractive or unattractive objects.

7. OCC Cognitive Theory of Emotions • Goal-based emotions: Depend on the desirability of an unconfirmed, confirmed or disconfirmed event, whether it was anticipated, and whether it happened to the agent itself or someone else (joy, distress,hope, fear, satisfaction, disappointment,relief, fears-confirmed,happy-for, resentment,gloating, pity). • Standard-based emotions: Depend on the praiseworthiness of the action of an agent as the result for an effected standard (pride, shame, admiration, reproach). • Attitude-based emotions: Depend on the appealing ness of an object (love, hate).

8. The OCC Model

9. Virtual Reality Game • The popularity of software games among children and adolescents may be exploited for educational purposes. Virtual reality games constitute a very promising mean of developing more attractive educational applications. • Such gaming applications, which typically provoke a wealth of emotions to users, can become an advanced test bed for affective states. • The test bed for our research is an ITS for teaching English orthography and grammatical rules. This ITS operates as a virtual reality game, and is called VIRGE (Virtual Reality Game for English).

10. Virtual Reality Game • Students have the opportunity to play a 3D game, similar to the commercial ones, which enables them to learn while playing. The student must fight his/her way through a maze by using his/her domain knowledge. • It is an educational application that models aspects of student behaviour, by combining evidence from students’ cognitive and detectable behavioural characteristics. • Recognises evidence about important students’ emotions and provides appropriate feedback.

11. Animated agents • When users interact with a computer, they provide a great deal of information about themselves. • Agents have been quite successful at observing users’ behavior and they have been used in learning environments in order to capture the users’ characteristics and perform user-modeling tasks. • VIRGE communicates with the student via three types of animated agent, the virtual enemy, the virtual advisor and the virtual companion.

12. The Virtual Enemy of the Game

13. The Virtual Advisor of the Game

14. Animated agents • The animated agent who acts as an advisor, has the form of a female angel, and appears in situations where the student has to read new parts of the theory or has to repeat parts that s/he appears not to know well. • The virtual enemy agent is a dragon guard outside every door of the game, and opposes himself as an obstacle on the student’s course during the game, by asking questions. • The virtual companion appears, and makes some remarks in a casual way as if a friend was talking to the student. He is responsible for showing empathy to the students and help them in managing their emotions while playing.

15. Students’ Goals and Standards while playing the game At first we were able to record some of the students goals and standards while using the virtual reality educational game: • Goals depending on: Prospect Relevant Events: Prospect Irrelevant Events: - Being Correct - Have Fun - Avoid Mistakes - Avoid Confusion - Finish Quickly - Learn • Standards depending on : • Self Agent Actions: Other Agent Actions : • - Be a very good student - Being Helped • - Not being a bad student - Not Being Disturbed

16. Connection of Students’ characteristics with their Goals and Standards • The student characteristics that are being modelled concern cognitive characteristics of students (answers’ results - errors) as well as behavioural actions while learning (user actions), that are connected to their goals and standards, and can lead to emotions. • Behavioural characteristics: that are reactions to events or actions of agents and can be connected to the students’ goals and standards for the game. • Cognitive characteristics: that is evidence on the students’ degree and quality of knowledge of the parts of the lessons that are examined during the game, and are related to the students’ goals and standards for the game.

17. Example of a behavioural characteristic • Speed of answering: The time that it takes to the student to answer a question. In the case this time is very small it might mean that the student is having a good time playing and answers quickly to see more, so is connected to the Prospect Irrelevant goal of the student “Have Fun” and can lead to the emotion of Joy. For the same case the time is connected to the Prospect Relevant goal of the student “Finish Quickly” and can lead to the emotion of Satisfaction. On the opposite case that this time is really high, then this can reveal the student’s hesitation to answer and is connected to the Prospect Relevant goal of the student “Avoid Mistakes” and can lead to the emotion of Fear. • There are more such behavioural characteristics that can be connected to student goals and standards, and eventually lead to emotions.

18. Examples of cognitive characteristics • Error frequency: The percentage of errors among the answers given to all the questions so far. If the student has made a high degree of errors for his answers so far, then this characteristic is related to the standard of the student for Self Agent Actions “Not being a bad student” and can lead to the emotion of Shame. • Error persistent occurrence: Consecutive errors for the last questions or for a specific orthography or grammatical rule. This characteristic is connected to the Prospect Relevant goal of the student “Being Correct” and can lead to the emotion of Disappointment.

19. Adaptation of the OCC Theory • We were able to identify students’ goals and standards during their interaction with our educational software. • Observable behavioural characteristics and measurable cognitive characteristics were tracked down, that are reactions to events and actions of agents, either for self-agent or the application’s agents, and are connected to students’ goals and standards. • These characteristics are the intensity variables for the desirability of an event and the praise-worthiness of an action of an agent, so are linked to the intensity of the corresponding emotions according to the OCC cognitive theory model.

20. Adaptation of the OCC Theory The subsection chosen from the OCC model focuses on the Prospect Based, Well-Being, and Attribution emotional categories of the original OCC model.This model refers to 12 of the 22 OCC emotion categories: • Reactions of the subject to an event which is relevant to his/hers goals depending on the desirability of the unconfirmed, confirmed or disconfirmed event, and whether it was anticipated, (joy, distress,hope, fear, satisfaction, disappointment,relief, fears-confirmed). • Reactions of the subject to an action of an accountable agent (including itself) depending on the praiseworthiness of the action of the agent as the result for an effected standard (pride, shame, admiration, reproach).

21. Intensity of an Emotion • The intensity of an emotional state is very important for the selection of the appropriate advice for the user. • The combination of such emotional states, either positive or negative, can provide information about the general mood of the student (Happiness, Sadness) and lead to affective computing.

22. Adaptation of the OCC Model

23. OCC Theory of emotions • The OCC theory of emotions suggests that for the purpose of finding out if an emotion really occurred to an individual there is a need for the specification of a specific value that is called the threshold value. • If the potential value of an emotion is lower than the threshold value then the individual is not considered to experience the emotion. Otherwise the intensity of the emotion experienced is the difference between the potential value of the emotion and the threshold value.

24. Using OCC Theory • We have used the OCC theory to find out which of the characteristics of the student that are the intensity variables for the emotional states of each student, have a value that is significant enough for the emotion to occur. • The threshold value of each intensity variable is calculated, by taking into account the mean value and the standard deviation value of each intensity variable for each individual student. • This decision was based on the fact that it is important to know if an intensity variable (cognitive or behavioural characteristic) is out of its usual bounds for the student.

25. Example of the intensity of an intensity variable leading to an Emotional state • For example if the value of the behavioural characteristic of speed of answering in a particular answer is greater than its threshold value (outside its usual bounds) then the intensity for this intensity variable is: • Intensity (speed of answering) := Value – Threshold_Value (speed of answering). • The Speed of answering characteristic, which in this case is related to the goals “Having Fun” and “Finish Quickly”, is linked to the emotional categories of Joy and Satisfaction. So the intensity of these emotions is equal to the intensity of the intensity variable.

26. An Example of advice • If a student is hasty, which is a result of its Speed of answering characteristic characteristic intensity, and makes a lot of errors receives the following advice from the companion agent: • “ You seem to be quite anxious about answering. There is no need to feel that way. Take your time to think before giving an answer “.

27. Conclusions • We have described how evidence from the students’ behavioural and cognitive characteristics may be combined with the OCC theory of emotions, for drawing inferences for the student’s emotional state while interacting with the educational application. • It is very important for a intelligent tutoring system to be flexible, depending not only on the learning model of a student but also on the affective model. • Affective computing needs a lot of work and evaluations as to be able to provide safe conclusions.