1 / 34

Sensing Valence and Confusion with Facial EMG

Sensing Valence and Confusion with Facial EMG. Phil Davis and Hsin-Ni Ho. Outline. Experimental Setup Valence Problems Addressed Experimental Procedure Results Discussion Confusion Problems Addressed Experimental Procedure Results Discussion Conclusions. Outline.

austine
Download Presentation

Sensing Valence and Confusion with Facial EMG

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sensing Valence and Confusion with Facial EMG Phil Davis and Hsin-Ni Ho

  2. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  3. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  4. Experimental Setup • Electrodes attached to: • Corrugator supercilii (“brow”) • Zygomaticus major (“cheek”) • Grounded on arm

  5. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  6. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  7. Problems Addressed • Can we sense valence from facial electromyographic (EMG) activity? • Intensity? • Can we sense valence from mouse behavior (pressure in particular)? • In general, what types of activity can we sense with facial EMG?

  8. Relevant Prior Work • Recent study by Larsen, Norris, Cacioppo • Change in mean “cheek” EMG activity correlated positively with valence • Change in mean “brow” EMG activity negatively correlated with valence (CS = “brow” muscle, ZM = “cheek” muscle)

  9. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  10. Experimental Procedure • 6 subjects • General procedure: • Subject watches a short film clip • Subject answers questions about film content and experienced affective state • Subject uses pressure sensitive mouse • EMG output is recorded during all stages

  11. Film Clip Sequence • Each film clip was roughly 3 minutes • Golf Instruction (Neutral) • Clip from “The Champ” (Negative) • Sadness in 94.2% (Gross & Levenson) • Golf Instruction (Neutral) • Clip from “Robin Williams Live” (Pos) • Amusement in 84.1% (Gross & Levenson) • Did not vary order

  12. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  13. Self-reported Valence Consistent with Expectations • Neutral Valence: 4 subjects indicated “neutral” affect, 2 subjects indicated low intensity positive affect • Negative Valence: All 6 reported sadness • Positive Valence: 5 of 6 reported amusement • The sixth subject did not like Robin Williams

  14. Example of Output • Output normalized due to large scale differences

  15. Mean Output Consistent with Prior Work

  16. Individual Results Varied

  17. Other Observations • For some subjects, brow activity was a good indicator of web form activity

  18. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  19. Discussion • Highly sensitive to individual subject differences and electrode positioning • Mean output consistent with Larsen, Norris, Cacioppo • Do results extend to other positive/negative states? • Unable to correlate EMG output with self-reported mood intensity • EMG output may be useful as an activity recognition sensor • Unable to correlate mouse pressure or velocity with valence • Dynamic model of mouse behavior may produce better results?

  20. Outline • Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  21. Outline • Introduction and Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  22. Problem Addressed • Does the facial expression ‘frowning’ represent the feeling of confusion? • Can we recognize the feeling of confusion with the EMG outputs? • Does the importance of understanding influence the feeling of confusion?

  23. Outline • Introduction and Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  24. Experimental Procedure • 6 subjects • General procedure: • Subject listens to an audio clip • Subject answers questions about audio content and experienced affective state • EMG output is recorded during all stages

  25. Eliciting Confusion with Audio Recordings Two levels of the importance of understanding: • Low level importance of understanding. • High level importance of understanding. Audio clips to induce different levels of confusion:

  26. Ratings and Measurements for feeling of Confusion • Subjective rating • 5 scale Self-report confusion rating • Objective rating • Test of understanding • Measurement • EMG responses

  27. Outline • Introduction and Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  28. Result: EMG responses

  29. Result: Average EMG responses For brow • More EMG activity when • filling the web form • listening to clips with accent • EMG in ‘web-filling’ parts increase with level of confusion in high importance of understanding For cheek • EMG activity increase with degree of confusion ->Subject started to laugh

  30. Result:Self confusion rating and test score • create the feeling of confusion consistently Average plot

  31. Outline • Introduction and Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  32. Discussion • Huge individual difference in EMG responses -> normalize the data • More Brow activity when filling web forms -> It is more correlated to ‘the process of thinking about confusion thing’ • More Brow activity when listening to clip with accent • Cheek activity increase with level of confusion -> So confused that subjects were giving up • Difference in importance of understanding

  33. Outline • Introduction and Experimental Setup • Valence • Problems Addressed • Experimental Procedure • Results • Discussion • Confusion • Problems Addressed • Experimental Procedure • Results • Discussion • Conclusions

  34. Conclusions • Baseline difference in EMG for different muscles • Individual difference in EMG response • EMG response sensitive to electrode positioning • Brow EMG is negatively correlated with valence and positively correlated with feeling of confusion • Cheek EMG is positively correlated with valence • Facial EMG may be useful for activity recognition

More Related