1 / 50

Sensing Affective Experience

Sensing Affective Experience. Jennifer Healey Advanced Technology Group, Cambridge, MA Intel Massachusetts. Sensing Affective Experience. Physiological variables that reflect affective changes Labeling affective experience Frameworks of basic emotions Methods for quantifying emotion

joshua-cantrell
Download Presentation

Sensing Affective Experience

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sensing Affective Experience Jennifer Healey Advanced Technology Group, Cambridge, MA Intel Massachusetts

  2. Sensing Affective Experience • Physiological variables that reflect affective changes • Labeling affective experience • Frameworks of basic emotions • Methods for quantifying emotion • Sensing Affect in the Laboratory • Eight emotion experiment • Affective DJ • Sensing Affect in the Field • Ambulatory Wearable Monitoring • StartleCam • Sensing Affect in an Automobile • Conclusions

  3. Sensing Affective Experience • Physiological variables that reflect affective changes • Labeling affective experience • Frameworks of basic emotions • Methods for quantifying emotion • Sensing Affect in the Laboratory • Eight emotion experiment • Affective DJ • Sensing Affect in the Field • Ambulatory Wearable Monitoring • StartleCam • Sensing Affect in an Automobile • Conclusions

  4. Physiological Variables • Heart rate • BVP (+ vasoconstriction) • Heart rate variability (HRV) • ECG (heart rate and HRV) • Skin conductance (GSR) • Electromyogram (EMG) • Respiration

  5. Blood Volume Pulse

  6. Electrocardiogram (ECG)

  7. Skin Conductance (GSR) hand toes

  8. Respiration (through chest cavity expansion)

  9. Electromyogram (EMG)

  10. Labeling Affective Experience • Basic Emotions • Emotion Models • Measuring response • Assumed from experimental design • Questionnaire • Third party observer

  11. Basic Emotions

  12. Emotion Models Arousal-Valence Circumplex cute baby erotic flowers ski jump basket snake mutilated face cemetery

  13. Labeling Challenges • Questionnaires considered the gold standard • People are notoriously bad at filling them out • Hard to put emotions on a scale • The person either has to be interrupted while experiencing the emotion or recall the emotion after some time lag • Third party observer • Might not know the person • Better if it is a close friend • Couples review video together (Marci) • Video used to trigger memories

  14. Sensing Affect in the Laboratory • Controlled conditions • Stationary subjects • Expected interactions • Generating affective response • Limited by modern ethical guidelines • Time frame usually small • Two Experiments • Clynes Sentic Cycle – Eight Emotions • Affective DJ

  15. Sentic Cycle Experiment • Designed by pianist/psychologist Manfred Clynes • Based on eight emotions for which he found unique finger pressure patterns: no emotion, anger, hate, love, sex, joy, reverence • Each emotion acted out over thirty days each day at the same time by the same subject

  16. Sentic Experiment Results • Anger could be differentiated with 100% sensitivity and 98% specificity • High vs. Low arousal emotions could be differentiated with 86% sensitivity and 88% specificity • Positive vs. Negative emotions could be differentiated with 82% sensitivity and 50% specificity

  17. Affective DJ • An application for personalized music selections on skin conductance readings • A library of songs is stored on the wearable, classified by a five level arousal rating • Songs were randomized within arousal category • Next song selection: • GSR increase -> a lower arousal category • GSR decrease -> higher arousal category

  18. Affective DJ Results • Subjects showed no significant preference for Affective DJ over random • Problems • Lack of song variety • Long “baselining” period • Skin conductance response seemed more correlated to song volume than arousal rating

  19. Wearable Sensor System • Wearable computer with multiple sensors: • Skin Conductivity (GSR) • Electromyogram (muscle activity) • Blood Volume Pulse (heart rate) • Respiration (chest cavity expansion) • Wired connections for highest signal quality • Standard sensor placements • Displays (Head mounted or PDA) for signal monitoring • Keyboard or PDA for annotations • Digital Camera for annotations

  20. Sensing Affect in the Field • Uncontrolled conditions • Ambulatory subjects • Unpredictable interactions • Authentic affective response • Responses not noted by subjects • Labeling system not sufficiently restricted • Physical physiological response and feelings overwhelmed affective physiological response and feelings • Two Experiments • Free ambulatory monitoring • StartleCam

  21. Wearable System (original)

  22. Palm Pilot Interface • Interface: Connected via • Serial Cable to the main • wearable unit • Check Signals • Take Notes • Run Scripts • Select Music • Activate Camera

  23. Wearable systems PDA-based www.thoughttechnology.com

  24. PDA Interface • Interface: Connected via Bluetooth to sensors • Check Signals • Take Voice Notes • Checklist Interface for annotations • Run Scripts • Record ECG and Accelerometer data

  25. Data Collected • Poorly annotated • Confounded with physical activity • Constant problems with sensor failure

  26. Electromyogram Skin Conductivity

  27. Respiration Blood Volume Pulse

  28. Skin Conductivity Skin Conductivity Blood Volume Pulse Respiration

  29. Physiological reactions to activity Resp HR

  30. Respiration Signal During Talking

  31. Data Annotation • Intially user initiated • Palm Pilot notes • User controlled camera that only captured images when prompted • Automatic continuous image capture

  32. Drinking Coffee

  33. StartleCam • Continuously record images from wearable camera • Simultaneously record skin conductance signal • Run real-time “startle” detection algorithm • Evaluate response magnitude • Send images to server (SafetyNet)

  34. StartleCam Movie

  35. Sensing Affect in an Automobile • Too many variables in the ambulatory environment • Limit scope of affect: stress • Document with video • Experimental design: rest, city, hwy • Validate with questionnaires

  36. Automotive System

  37. Results of the Driving Experiment • Questionnaire results, video coding confirmed model • Recognition: • rest(36/36), city(37/39), hwy (36/37) • Best correlated features: • Mean GSR, LF/HF ratio, HR • Best for discrimination: • Mean respiration, mean GSR, startle characteristics, mean EMG

  38. Conclusions • Sensing Affect in the laboratory allows for tight control and labeling of affective data, however responses may not have the same strength as when felt spontaneously in the field • Sensing Affect in the field is difficult both because of confounding ambulatory artifacts and due to poor compliance from subjects for labeling affective data • Sensing Affect in a semi-controlled environment such as an automobile an automobile provides a compromise solution where authentic responses can be captured with fewer confounding variables.

  39. Questions?

More Related