1. Monitoring Emotions While Students Learn with AutoTutor Discriminability and Diagnosticity of AUs and Emotions Bethany McDaniel August 31, 2005 Grant funded by the National Science Foundation

2. Where we are… • 2003 - Emote-Aloud Study (handout) • 2004 - Gold Standard Study (handout) • 2005 - Speech Recognition Study (will be covered in October by Patrick Chipman)

3. Discriminability and Diagnosticity • Discriminability • How well can each Action Unit (AU) be detected • Motion, Edge, and Texture • Diagnosticity • How frequent an AU is with one emotion compared to the frequency of the AU across all emotions.

4. Discriminability • Rate AUs on motion, edge, and texture • Motion: How detectable the change is from the Neutral position. • Edge: How clearly defined a line or object on the face is in relation to the surrounding area. • Texture: The level of graininess for the general area. The degree of variation of the intensity of the surface, quantifying properties such as smoothness, coarseness and regularity.

5. Grading Discriminability • 2 expert judges • trained on the Facial Action Coding System (Ekman & Friesen, 1978) • Rate Motion, Edge, and Texture on a 1-6 scale • 1=Very Difficult, 6=Very Easy • Averaged the 3 scores for each expert • Score between 3-18 • 3=Very Difficult to detect, 18= Very Easy to detect

6. In the following table, rate the given dimensions (Motion, Edge, & Texture) on the following 6-point scale Motion, Edge, and Texture 1 Very Difficult 2 Difficult 3 Moderately Difficult 4 Moderately easy 5 Easy 6 Very Easy

7. Results

8. Diagnosticity • Looked at AUs that had been identified with certain emotions (Craig et al., 2004) • Frustration: AUs 1,2, and 14 • Confusion: AUs 4,7, and 12 • Boredom: AU 43 • Formula: p (AU| emotion x) – p (AU| all emotions) 1 - p (AU| all emotions)

9. Frustration

10. Frustration

11. Confusion

12. Confusion

13. Boredom

14. Boredom

15. Diagnosticity

16. Discriminability x Diagnosticity