Grant funded by the National Science Foundation

1. Monitoring Emotions While Students Learn with AutoTutor Discriminability and Diagnosticity of AUs and Emotions Bethany McDaniel August 31, 2005 Grant funded by the National Science Foundation

2. Where we are… • 2003 - Emote-Aloud Study (handout) • 2004 - Gold Standard Study (handout) • 2005 - Speech Recognition Study (will be covered in October by Patrick Chipman)

3. Discriminability and Diagnosticity • Discriminability • How well can each Action Unit (AU) be detected • Motion, Edge, and Texture • Diagnosticity • How frequent an AU is with one emotion compared to the frequency of the AU across all emotions.

4. Discriminability • Rate AUs on motion, edge, and texture • Motion: How detectable the change is from the Neutral position. • Edge: How clearly defined a line or object on the face is in relation to the surrounding area. • Texture: The level of graininess for the general area. The degree of variation of the intensity of the surface, quantifying properties such as smoothness, coarseness and regularity.

5. Grading Discriminability • 2 expert judges • trained on the Facial Action Coding System (Ekman & Friesen, 1978) • Rate Motion, Edge, and Texture on a 1-6 scale • 1=Very Difficult, 6=Very Easy • Averaged the 3 scores for each expert • Score between 3-18 • 3=Very Difficult to detect, 18= Very Easy to detect

6. In the following table, rate the given dimensions (Motion, Edge, & Texture) on the following 6-point scale Motion, Edge, and Texture 1 Very Difficult 2 Difficult 3 Moderately Difficult 4 Moderately easy 5 Easy 6 Very Easy

7. Results

8. Diagnosticity • Looked at AUs that had been identified with certain emotions (Craig et al., 2004) • Frustration: AUs 1,2, and 14 • Confusion: AUs 4,7, and 12 • Boredom: AU 43 • Formula: p (AU| emotion x) – p (AU| all emotions) 1 - p (AU| all emotions)

9. Frustration

10. Frustration

11. Confusion

12. Confusion

13. Boredom

14. Boredom

15. Diagnosticity

16. Discriminability x Diagnosticity