Understanding Development Methods From Other Industries to Improve the Design of Consumer Health IT Enid Montague, PhD

1. Understanding Development Methods From Other Industries to Improve the Design of Consumer Health ITEnid Montague, PhD Agency for Healthcare Research and Quality Task Order No. 10 Contract No. HHSA2902009000231

2. Project Oversight • AHRQ Task Order Officer: Teresa Zayas-Cabán • AHRQ Alternate Task Order Officer: Kevin Chaney • Project Director: Enid Montague

3. Project Team • Jennifer Crafts (Westat) • David Damico (Westat) • Patricia FlatleyBrennan (University of Wisconsin- Madison) • Enid Montague (Northwestern University) • RituAgarwal, Center for Health Information and Decision Systems (CHIDS) (U of MD) • P.K. Kannan, Center for Health Information and Decision Systems (CHIDS) (U of MD) • Kenyon Crowley, Center for Health Information and Decision Systems (CHIDS) (U of MD)

4. Background and Impetus for Project • Slow adoption of consumer health IT • New user-centered design approaches for development of consumer products are emerging in a variety of industries • Success of product development approaches, methods, and philosophies in other industries could generalize to personal health care

5. Project Objectives • Identify new and emerging practices for developing consumer products that may generalize to designing consumer health IT applications. • Conduct environmental scan and grey literature review to Identify relevant tools and design methods • Interview key informants about design processes

6. Objective One • Conduct environmental scan and grey literature review to Identify relevant tools and design methods

7. Environmental Scan and Grey Literature Review: Method Overview Recommended Design Methods for Categories of Consumer Health IT Applications

8. Environmental Scan and Grey Lit Review: Findings • Considerable variety in methods used across products; common themes are captured • Choice of method(s) likely influenced by the nature of the consumer health IT application • Development of a hybrid product (integrated physical device and digital component as a single product from the manufacturer, e.g. Fitbit) vs. development of a purely digital product (e.g., WebMD) • Limitations of study: • Design method is just one of many factors in product success • Focus of review was design methods using an engineering approach

9. Environmental Scan and Grey Lit Review: Overview of Findings • Observed many methods used by successful products; applying, disseminating and incentivizing use of good design may be key point • Generally for all consumer products: • Used methods that incorporate high levels of user involvement and iteration • Used one or more of the most frequently used design methods • Design teams contained medium breadth with regard to size and skill-sets • Engaged human factors experts • Strived to “Keep it simple” – parsimonious feature set • Paid careful attention to user characteristics

10. Environmental Scan and Grey Lit Review: Overview of Findings (cont’d) • Additional design considerations for hybrid consumer products: • Used multiple design methods • Used prototypes and consumer feedback based on their use

11. Objective Two • Interview key informants about design processes

12. Key Informant Interviews: Approach and Challenges • Approach • Challenges • Company restrictions on employees’ participation in research • Availability of all original key informants • Rapid turnover within consumer design field

13. Key Informant Interviews:Selection Criteria

14. Key Informants’ Products and Companies

15. Key Informants Interviewed • Company size range: 4 – 50,000 • Roles and titles • Design, User Experience, or Human Factors • User Interaction Designer • Senior Designer • Senior User Interaction Designer • Human Factors Engineer • User Research Lead • Management • Senior VP of Product and Program Management • CEO and Founder • Director, Innovation and Advanced Technology • Co-founder

16. Methods • Used interview guide to conduct interviews • Recorded and transcribed interviews • Analyzed transcripts using NVivo software and a thematic analysis method • Coded interviews based on the key elements in the interview guide (idea generation, identifying end users, concept development, testing, usefulness, strategies influencing commercial success) • Analyzed transcripts for overall themes

17. Key Informant Interviews:Overview of Primary Themes

18. Key Informant Interviews: Overview of Primary Themes (cont’d)

19. Key Informant Interviews: Secondary Themes • Secondary themes: • Designing for entertainment and experience versus usability and functionality • Leveraging people relationships in product design or business strategies

20. Areas for Future Research • Building evidence base of effective consumer health IT application design methods may be addressed by: • Comparing alternative levels of user involvement and iteration for different types of consumer health IT applications. • Isolating design processes currently in use for consumer health IT applications. • Developing case studies retrospectively documenting design processes of market leaders of consumer health IT applications. • Identifying contingencies that may affect the efficacy of different design methods for different user populations. • Investigating user response to products that are modular in nature.

21. For More Information • http://healthit.ahrq.gov/developmentmethodsbackgroundreport • http://healthit.ahrq.gov/developmentmethodskeyinformantinterviewsreport

22. Acknowledgements • Technical Expert Panel members: • Tony Andre (Interface Analysis Associates) • Deborah A. Boehm-Davis (George Mason University) • Sara Czaja (University of Miami, CREATE) • Judith Gregory (IIT Institute of Design) • Seth Howard (Epic) • Tony Hu (American Innovative, LLC) • Rita Kukafka (Columbia University)

23. THANK YOU! For additional information please feel free to contact me or visit http://healthit.ahrq.gov/ www.enidmontague.com