Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSense

1. Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSense Santosh Kumar Computer Science, University of Memphis

2. Our Team Behavioral Science Engineering • Dr. Mustafa al’Absi, UMN • Dr. J Gayle Beck, Memphis • Dr. David Epstein, NIDA, NIH • Dr. Tom Kamarck, Pittsburgh • Dr. Satish Kedia, Memphis • Dr. Kenzie Preston, NIDA, NIH • Dr. Marcia Scott, NIAAA, NIH • Dr. Saul Shiffman, Pittsburgh • Dr. Annie Umbricht, Johns Hopkins • Dr. Kenneth Ward, Memphis • Dr. Larry Wittmers, UMN • Dr. Anind Dey, CMU • Dr. Emre Ertin, Ohio State • Dr. Deepak Ganesan, UMass • Dr. Greg Pottie, UCLA • Dr. Justin Romberg, Georgia Tech • Dr. Dan Siewiorek, CMU • Dr. Asim Smailagic, CMU • Dr. Mani Srivastava, UCLA • Dr. Linda Tempelman, Giner Inc. • Dr. Jun Xu, Georgia Tech Santosh Kumar, University of Memphis

3. Students & Postdocs Memphis CMU, OSU, UCLA, Georgia Tech., UMN • Dr. Andrew Raij (now at USF) • Dr. Kurt Plarre • Dr. Karen Hovsepian • Amin Ahsan Ali • Santanu Guha • Monowar Hussain • Somnath Mitra • Mahbub Rahman • Sudip Vhaduri • Dr. Motohiro Nakajima, UMN • Patrick Blitz, CMU • Brian French, CMU • Scott Frisk, CMU • Nan Hua, Georgia Tech • Taewoo Kwon, OSU • Moaj Mustang, UMass • Siddharth Shah, OSU • Nathan Stohs, OSU Santosh Kumar, University of Memphis

4. Paradigm Shift in Disease Prevalence • Infectious diseases, and those from poor hygiene & nutrition not as prevalent • They are replaced by diseases of slow accumulation • Heart diseases • Cancer, Ulcer • Depression, Migraine Santosh Kumar, University of Memphis

5. Growing Epidemic – Stress & Addiction • Stress & addictive behavior lead to or worsen diseases of slow accumulation • Stress: headaches, fatigue, heart failures, hypertension, depression, addiction, anxiety, rage • Smoking: cancer, lung diseases, heart diseases • Yet, both continue to be widespread • Stress:43% adults suffer adverse health effects • Smoking:responsible for 20% of deaths in US • An urgency to help individuals reduce stress & abstain from addictive behavior Stress costs $300 billion/yr Smoking costs $193 billion/yr Santosh Kumar, University of Memphis

6. Addressing Stress & Addiction • An unobtrusively wearable sensor suite called AutoSense • So, individuals can wear it in natural environment • Robust inference of stress from physiological measures • Automatically measure physiological and psychological stress • Automatic inference of addictive behaviors • Smoking, drinking, drug usage from sensor measurements • Detect addiction urges to provide timely intervention • Craving for smoking and drug usage • Contexts/cues that may lead to craving and eventual relapse • Infer other moderating behavioral & social contexts • Conversation, physical activity, traffic stressors, etc. Santosh Kumar, University of Memphis

7. Outline • Hardware and Software Platforms • AutoSense sensor suite • FieldStream mobile phone framework • Inferring Stress • Detecting stress from physiology • Predicting perceived stress • Ongoing User Studies • Detecting smoking, drinking, craving, drug usage, etc. • Roadmap & Long-term Vision Santosh Kumar, University of Memphis

8. AutoSense Wearable Sensor Suite Chestband sensors: ECG, Respiration, GSR, Ambient & Skin Temp. , Accelerometer Armband sensors: Alcohol (WrisTAS) , GSR, Temp., Accelerometer Android G1 Smart Phone Santosh Kumar, University of Memphis

9. Key Features of AutoSense Hardware • Ultra low power • Six sensors (ECG, GSR, Resp., Temp, Accel) consume 1.75 mA • Overall current consumption < 3mA (for 10+ days of lifetime) • Sampling and transmission of 132 samples/sec (i.e., 1.8 kbps) • Reliable radio • ANT with integrated quality of service and duty cycling • Reliable and timely wireless transmissions in crowding scenarios • Antenna impedance is matched for human body • Power loss reduced from 33% (for free space configuration) to 0.1% • Operates at 2480-2524 MHZ band to be immune to Wi-Fi • Average packet loss rate of 0.57% even when Wi-Fi activity is intense Santosh Kumar, University of Memphis

10. FieldStream – Mobile Phone Framework • For use in conducting scientific user studies • In both supervised lab settings and in uncontrolled field settings • It collects measurements • Sensor measurements from wearable and phone sensors • Self-reports from subjects • Computes tens of features and various statistics over them (e.g., HR, HRV, RR, Minute Ventilation) • Makes inferences using machine learning algorithms • Stress, posture, activity, conversation, and commuting • Detects sensor detachments and loosening • Is reconfigurable • So, no need for change in source code for use in a new user study Santosh Kumar, University of Memphis

11. Computes base features (e.g., R-R interval) & statistics over them Provides a common interface to all sensors & populates buffers for feature computation Converts stream of sensor measurements into packets & delivers to intended recipient Santosh Kumar, University of Memphis

12. Deployment Experiences and Findings • 21 subjects in UMN - completed • Lab session on stress; 10-14 hours per day for 2 days in field • 36 subjects in Memphis - completed • 3 consecutive days in field with daily visits to the lab Some findings on human behaviors in our subject pool • Stress occurrence in daily life (Plarre et. al., in ACM IPSN’11) • Subjects were psychologically stressed 26-28% of time • Natural conversations (Rahman et. al., in ACM Wireless Health’11) • Frequency of conversations : 3 per hour • Avg. duration of a conversation: 3.82 minutes • Avg. Time between conversations: 13.3 minutes Santosh Kumar, University of Memphis

13. Outline • Hardware and Software Platforms • AutoSense sensor suite • FieldStream mobile phone framework • Inferring Stress • Detecting stress from physiology • Predicting perceived stress • Ongoing User Studies • Detecting smoking, drinking, craving, drug usage, etc. • Roadmap & Long-term Vision Santosh Kumar, University of Memphis

14. Measuring Stress in the Field • Self-reports have been used for a long time • Questionnaires or surveys • Measures perceived stress • Strengths and limitations • (+) Captures detailed information • (+) Proximal predictor of mental health • (-) Distal predictor of physical health • (-) Discrete sampling • (-) Burden to participant • Need an automated approach for continuous stress measurement in the field Santosh Kumar, University of Memphis

15. Continuous Measure of Stress • Can use physiological measurements to assess stress, but • Physiology is affected by several factors, not only stress • Activity, change in posture, speaking, food, caffeine, drink, etc. • How to separate out the changes in physiology due to stress? • How to map physiology to psychology? Santosh Kumar, University of Memphis

16. The Quest for Automated Stress Measure • Predicting psychological state from physiology • William James – pioneering work (1880) • John Cacioppo and others – revitalized interest (1990) • Several studies on emotion and stress prediction • Identified physiological markers of stress and emotion Example: Heart rate, skin conductance response • But, confined to controlled settings • Few studies in uncontrolled environments • M. Myrtek’96 , J. Healey’05, J. Healey’10, • Either no validated stressors, no lab session to train models, not able to account for confounders, or tried to match self-reports directly Santosh Kumar, University of Memphis

17. In the AutoSense Project • We developed a new wearable sensor suite • Conducted a scientific study with validated stress protocol • 21 participants, 2 hour lab study, 2 day field study • Protocoldesigned by behavioral scientists • Stressors used are validated and known to produce stress • Self-reports designed by expert behavioral scientists • Developed new stress models to measure • Physiological response to stress • To measure adverse physiological effects of stress • Perception of stress in mind • To derive a continuous rating of perceived stress Santosh Kumar, University of Memphis

18. Lab Study – Stress Protocol • 2 hour lab session • Subjects exposed to three types of stressors • Public speaking – psychosocial stress • Mental arithmetic – mental load • Cold pressor – physical stress • Physiological signals recorded at all times • Using AutoSense • Also, collected self-reported stress rating 14 times Public Speaking Mental Arithmetic Cold Pressor Baseline Recovery Start End 10 Min 10 10 4 4 4 5 4 4 5 4 5 4 4 10 10 10 Santosh Kumar, University of Memphis

19. Self-Report Measures of Stress • Self-report questions related to affective state Santosh Kumar, University of Memphis

20. Santosh Kumar, University of Memphis

21. Our Aproach Santosh Kumar, University of Memphis

22. Identified 22 Features from Respiration Basic Features Statistical Features Inhalation Duration Mean Exhalation Duration Median Respiration Duration 80th Percentile Insp./Exp. Ratio Quartile Deviation Stretch Breathing Rate Minute Ventilation Santosh Kumar, University of Memphis

23. Computed 13 Features from ECG Basic Features Statistical Features Variance Power in low/medium/high frequency bands RR Intervals Ratio of low frequency/high power Mean Median RSA 80th Percentile Quartile Deviation Santosh Kumar, University of Memphis

24. Feature and Classifier Selection • Used Weka for Training • Evaluated Decision Tree, DT with Adaboost, and Support Vector Machine • Using 10-fold cross validation, and training/test data • Classification results using 35 features • After feature selection, 13 features • 8 Respiration, 5 ECG Santosh Kumar, University of Memphis

25. Classification Accuracy on Lab Data Santosh Kumar, University of Memphis

26. Our Aproach Santosh Kumar, University of Memphis

27. Perceived Stress Model • Use a binary Hidden Markov Model • To reduce number of parameters, we approximate by •  models the gradual decay of stress with time •  models the accumulation of stress in mind due to repeated exposures to stress • Both  and  are person dependent and are learned from self-reported ratings of stress Santosh Kumar, University of Memphis

28. Evaluation of the Model (on Lab Data) • Correlation of perceived stress model and self-report rating in the lab session • Over 21 participants • Median correlation • 0.72 Santosh Kumar, University of Memphis

29. Field Study Protocol • Participants wore AutoSense continuously for 2 days • Going about their daily life (home, school, etc.) • Except when sleeping at night • Field self-reports • Participants responded to self-reports 20+ times each day • Same questions about affect state as in the lab • Additional context information • Additional behaviors automatically collected • Speaking, from respiration patterns • Physical activity, from accelerometer Santosh Kumar, University of Memphis

30. Realities of Natural Environment • Data eliminated • 37% affected by activity • 30% by bad quality • Less than 4 min consecutive data • 4 subjects missing data or self-report Santosh Kumar, University of Memphis

31. Evaluation of the Model (Field) • Compared average stress ratings over both days • Accumulation model versus self-report • Linear interpolation Santosh Kumar, University of Memphis

32. Outline • Hardware and Software Platforms • AutoSense sensor suite • FieldStream mobile phone framework • Inferring Stress • Detecting stress from physiology • Predicting perceived stress • Ongoing User Studies • Detecting smoking, drinking, craving, drug usage, etc. • Roadmap & Long-term Vision Santosh Kumar, University of Memphis

33. Ongoing User Studies • Memphis Study • 40 daily smokers and social drinkers • A lab study followed by one week in the field • Stress, drinking, smoking, and craving for cigarettes marked • National Institute on Drug Abuse (NIDA) Study • 20 drug addicts undergoing treatment • Two lab sessions and 4 weeks in field • Smoking, craving, and stress marked in lab; • Craving, stress, and drug usage reported in the field • Johns Hopkins Study • 10 drug addicts in residential treatment • Drug injection in lab, daily behaviors marked in the field • To develop detectors for smoking, craving, and drug usage Santosh Kumar, University of Memphis

34. Roadmap • The near-term goal is to develop personalized stress and addiction assistants on the mobile phone to • Help reduce stress, e.g., least stressful route for driving • Break addiction urges where and when they occur • But, these applications will impact someone’s health • Will it indeed be helpful to each user and not hurt anyone? • Will it help maintain healthy behaviors even after the novelty phase? • How do we generate evidence for its validity, efficacy, safety? • Within reasonable time and effort, unlike multiyear RCTs • How do we design it so it has greater chance of success? • Various theories exist (e.g., stages of change, social cognitive theory) • But, no overall theory for designing adaptive interventions exist today Santosh Kumar, University of Memphis

35. Long-term Vision • Use these experiences to discover the scientific principles that can be used broadly in mobile health (mHealth) • To design and develop • New mHealth measures that are robust enough for field usage • New mHealth treatments and interventions that work • To generate evidence of validity, efficacy, and safety of mHealth Contribute to the newly emerging science of mHealth Santosh Kumar, University of Memphis

36. Further Reading • E. Ertin, N. Stohs, S. Kumar, A. Raij, M. al'Absi, T.Kwon, S. Mitra, Siddharth Shah, and J. W. Jeong, “AutoSense: Unobtrusively Wearable Sensor Suite for Inferencing of Onset, Causality, and Consequences of Stress in the Field,” ACM SenSys, 2011. • Md. Mahbubur Rahman, Amin Ahsan Ali, Kurt Plarre, Mustafa al'Absi, Emre Ertin, and Santosh Kumar, “mConverse: Inferring Conversation Episodes from Respiratory Measurements Collected in the Field,” ACM Wireless Health, 2011. • Mohamed Mustang, Andrew Raij, Deepak Ganesan, Santosh Kumar and Saul Shiffman, “Exploring Micro-Incentive Strategies for Participant Compensation in High Burden Studies,” to appear in ACM UbiComp, 2011. • K. Plarre, A. Raij, M. Hossain, A. Ali, M. Nakajima, M. al'Absi, E. Ertin, T. Kamarck, S. Kumar, M. Scott, D. Siewiorek, A. Smailagic, and L. Wittmers, “Continuous Inference of Psychological Stress from Sensory Measurements Collected in the Natural Environment,” ACMIPSN, 2011. • Andrew Raij, Animikh Ghosh, Santosh Kumar and Mani Srivastava, “Privacy Risks Emerging from the Adoption of Inoccuous Wearable Sensors in the Mobile Environment,” In ACM CHI, 2011. Nominated for best paper award Nominated for best paper award Santosh Kumar, University of Memphis

37. Outline • Hardware and Software Platforms • AutoSense sensor suite • mStress mobile phone framework • Inferring Stress • Detecting stress from physiology • Predicting perceived stress • Ongoing User Studies • Detecting smoking, drinking, craving, drug usage, etc. • Privacy Issues in mHealth research Santosh Kumar, University of Memphis

38. Behavior Revelation from Sensors • Accelerometer & gyroscopes can be used to monitor activity level • Can infer movement pattern and place from these sensors • See SenSys’10 paper on AutoWitness • Could also infer epileptic seizures • Respiration sensor can be used for activity monitoring or estimating the extent of pollution exposure • Can use it to infer conversation, smoking, and stress • Inferring of public speaking episodes could even pinpoint the identity of the subject • Development of other behavioral inferences in progress Santosh Kumar, University of Memphis

39. How Concerned are Study Participants? • Conducted a 66 subject (36 in NS) study • Evaluated their concern level as their personal stake in the data is increased • Also, how their concern level changes as modalities are added/removed Santosh Kumar, University of Memphis

40. Awareness & Concern • Sharing of stress, commuting, and conversation generate higher concern than the sharing of place Santosh Kumar, University of Memphis

41. Effect of Privacy Transformations • Disassociating time is more critical than disassociating place of occurrance • Even reducing timestamp to duration helps Santosh Kumar, University of Memphis