Understanding user engagement with digital interventions

1. Understanding user engagement with digital interventions Dr Jason Rentfrow, Dr Leanne Morrison, & Dr Sharon Lin On behalf of the UBhave, Emotion sense, LifeGuide, and POWeR teams

2. Understanding user engagement with power: digital intervention for weight management Leanne Morrison University of Southampton

3. How can we encourage better engagement with online interventions? How do users feel about these forms of supplemental support? What impact do they have on engagement and health-related outcomes? STUDY 1: Role of supplemental human support (“coaching”) • RCT of online POWeR programme + supplemental telephone coaching across communities in North East England (n = 786) STUDY 2: Role of supplemental mobile support (“apps”) • Series of in depth mixed methods ‘N-of-1’ case studies to explore engagement with and impact of online POWeR programme + POWeR Tracker app (N = 13)

4. POWeR Tracker: Android smartphone application to accompany the online programme • Maintain awareness of personal POWeR goals • Monitor progress Theory and evidence-based online programme to support users to adopt a sustainable and positive approach to weight management Developed using LifeGuide

5. Coaching Protocol: 2 short phone calls from a ‘POWeR coach’ in week 1 and week 4 Mohr et al. (‘Supportive Accountability’) Study 1: Community based RCT Consent and Registration N=1131 In collaboration with public health teams (Scott Lloyd, NHS Tees, NHS Durham and Darlington) Randomisation N=786 (69.5%) Coach N=247 (31.4%) Control N=275 (35%) Web only N=264 (33.6%) Is usage enhanced by the addition of brief human support in the form of telephone coaching? Qualitative interviews (n=19, purposively sampled) Responded to f/up N=162 (58.9%) Responded to f/up N=40 (15.2%) Responded to f/up N=53 (21.5%)

6. Did telephone coaching encourage usage? Participants in the coach arm significantly more likely to complete the core POWeR sessions

7. Perceptions of coaching • How did coaching help? • Praise and positivity • Accountability DEMOGRAPHICS: Older, lower health literacy, higher BMI, hypertension, previous referral to weight loss scheme PROGRESS: Greater weight loss USAGE AND ENGAGEMENT: More sessions completed, more log ins, more time spent online, Satisfaction with POWeR, fewer doubts about how to use POWeR, autonomous motivation • Low uptake • only 23.5% had one phone call, 18.6% had both

8. Study 2: POWeR Tracker • Variety of measures: • Daily questionnaire measures • Step counts (via pedometer) • Weekly telephone interviews • Objective data on web and app usage A: Week 1 B: Week 2 A: Week 3 B: Week 4 Q2. When, why and how do people engage with a web + mobile intervention? Q1. Does an app improve goal perceptions/progress? 13 participants followed over 4 weeks in series of n-of-1 case studies (ABAB vs. BABA) Compare web-based POWeRwith and without POWeR Tracker app

9. Did the POWeR Tracker app improve goal perceptions? “I guess if you weren’t like…with the questionnaires every day, um…if you weren’t doing the questionnaires every day then I think you would miss the app more…..Because…I guess the questionnaires every day were making you think about how well have I done today, or kept you motivated and yeah...it was just a period of evaluation. Whereas if you didn’t have those then you…yeah I think you would miss the app or miss the website more.” (Susan) No clear effect of app availability on daily goal perceptions or step count Measurement effect

10. Engagement with POWeR Tracker: When, where, why? Example: Dan • Short bursts of use ~ up to 10 minutes at a time • Use at key times (e.g. lunchtime – food choices, spare time in between lectures) • Response to app notifications Short bursts of on-the-go accessor time-relevant use Notifications prompted app use (when used) Use app primarily for a reminder of key information (e.g. food lists, goals) Variation in approach to using hybrid web + app intervention

11. Engagement with POWeR: Summary • Telephone coaching appears to improve engagement with online interventions and offer benefits to particular groups of user • Offering mobile tools or apps appears to improve the convenience and accessibility of health behaviour change interventions • Individual variation in tool preferences and patterns of use

12. ANALYSING COMPLEX DATA SETS Sharon Lin University of Southampton

13. Data analyses Visualization N-of-1 studies

14. Time spent on groups of pages

15. Re-ordered time spent on groups of pages – clustered time

16. Analyses of N-of-1 studies • To draw in the regression model: , )

17. A sample of Ubhave data Effect size = 0.006 α = 5592, β = 17.27, ρ(rho) = 0.06, σ = 2698

18. Statistical challenges of N-of-1 studies • Challenges • Small sample size • Autocorrelated errors in repeated measures arising from the individual under study • Non-normality of the responses • Remedy under investigations • Parametric bootstrap tests

19. N-of-1 Power Function Off (7D)-On(7D)-Off (7D)-On(7D) or ABAB

20. Tools will be available in the future • Functions for visualisation • Analysis tool for N-of-1 studies • correcting small sample and correlated data problems • Power functions for N-of-1 studies • giving guidance for N-of-1 study design

22. Acknowledgements Computer Scientists: Dr Cecilia Mascolo Dr Mark Weal Dr MircoMusolesi Dr DaniusMichaelides Dr Charlie Hargood Dr Neal Lathia Dr VeljkoPejovic Behavioural/Social Scientists: Professor Lucy Yardley Professor Susan Michie Professor Peter Smith Dr Jason Rentfrow Dr Leanne Morrison Dr Laura Dennison Dr Sharon Lin Funded by the EPSRC under the UBhave project For more information please visit: http://ubhave.org