Matthew Samore, MD VA Salt Lake City Health Care System Professor of Internal Medicine

1. Experience in improving healthcare decision-making with health IT: integrating theory, research, and practice Matthew Samore, MD VA Salt Lake City Health Care System Professor of Internal Medicine Adjunct Professor of Biomedical Informatics University of Utah

2. Acknowledgments • Michael Rubin, MD, PhD • Kim Bateman, MD • Brian Sauer, PhD • Lucy Savitz, PhD • Tom Greene, PhD • R. Scott Evans, PhD • Randall Rupper, MD, MPH • Salt Lake VA Informatics, Decision Enhancement, ,and Surveillance (IDEAS) Center Selected Investigators and Collaborators • Jonathan Nebeker, MD • Charlene Weir, PhD • Frank Drews, PhD • Partners: University of Utah, VA Salt Lake City, Healthinsight, CaduRx, Intermountain Healthcare • Funding: • VA HSR&D REA 08-264 • AHRQ R01 HS15413 • AHRQ 1R18HS017308-01

3. Thesis of this talk • Theory and models provide scientific underpinnings for generalization • Which supports comparative effectiveness research • For health services research and epidemiology: • Use of models understood • For clinical decision support: • Not so much • Health information technology ≠ informatics • Sub-disciplines such as cognitive informatics crucial

4. More succinctly expressed: • “The difference between theory and practice is that in theory there is no difference but in practice there is”

5. Statement of the problem: • “A disproportionate amount of literature on the benefits [of health information technology] that have been realized comes from a small set of early-adopter institutions that implemented internally developed health information technology systems..” Chaudhry et. al. Ann Intern Med. 2006;144:742-752

6. Addressing generalizability • In what contexts will effects generalize? • What accounts for variability in results? • Why are impacts lower in magnitude or narrower in scope in larger trials compared to single institution studies? • How to incorporate information about implementation, adoption, formative evaluation?

7. Relevance to comparative effectiveness research (CER) • CER priorities • Directly focused on health information technology • Compare the effectiveness of alternative redesign strategies—using decision support capabilities, electronic health records, personal health records • Indirectly tied to health information technology • Compare the effectiveness of various strategies • To control MRSA • To control healthcare associated infection • To enhance patients’ adherence to medication regimens

8. Addressing CER challenges • Need to explicitly formulate causal question • Determining identifiability • Defining level of inference • Validating methods to reduce bias

9. Conceptual frameworks (THEORY) • Natural & engineered systems • Co-evolution • Cognitive processing • Information overload ↔ fit-to-workflow • Cyclical models of control • Feedback and feed-forward

10. System co-evolution • Fundamental theorem in informatics • C. Friedman J Am Med Inform Assoc. 2009;16:169-170 • Proposed modification: • Computers plus humans create a distinct socio-technical system • Characteristics are not equivalent to other industries

11. Relevance • Level of inference needed to assess causal effect of health information technology: • Socio-technical system • Potential benefits of simulation

12. Cognitive processing • Motivation, mental models, tasks, goals • Influenced by social context • Lack of fit-to-workflow experienced as: • Information overload • Interruptions

13. Relevance • Cognitive informatics methods • Task analysis • Direct observation • Match implementation strategy to task complexity

14. Second law of thermodynamics as applied to cognition: • Humans seek states of reduced cognitive effort • Workarounds • As cognitive load increases, automatic processing systems kick-in

15. For those who believe that there is a Simpson’s quote for every situation • “In this house, we obey the laws of thermodynamics! • Homer Simpson’s response when his daughter builds a perpetual motion machine in which energy increases with time

16. Contextual Control Model • Feed-back systems not sufficient • Need to anticipate and predict • Pure feedback systems subject to loss of stability • Time horizon is long in strategic control modes • Relevance • Link between decision support and surveillance • Surveillance contributes feedback and feed-forward capabilities

17. Feedback & feed-forward decision support

18. Illustrative experience with decision support for antimicrobial prescribing • Two different technologies studied • Clinical task: • Management of patient with acute respiratory infection in outpatient setting • Whether or not to prescribe an antibiotic • Choosing the antibiotic • Diagnostic label • Impact of perceived or actual patient demand

19. Application of theory to practiceimplementation of electronic health records in rural settings • Socio-technical system • Hook was electronic prescribing • Stepwise approach to adoption • Accommodating variation • Readiness to change • Social context and clinic culture • Encouraging play • Avoiding information overload

20. Community intervention plus clinical decision support system • Standalone algorithms on handheld computers • Community randomized trial Effect on prescribing any antibiotic Samore MH et. al. JAMA. 2005 Nov 9;294(18):2305-14.

21. Clinical decision support system integrated with computerized clinic order entry • Algorithm usually triggered by ordering antibiotic • Clinic randomized trial Effect on macrolide prescriptions

22. Interpretation • Deciding whether to prescribe an antibiotic and choosing the drug involve different cognitive processes • Given that decision to prescribe an antibiotic is made • Possible to embed correct choice in workflow • Feed-forward decision support needed to impact the “is this a situation that warrants an antibiotic” decision • Relevant to drug-drug interaction alerting

23. Recommendations and conclusions • Models fundamental to translation of research into practice • Incorporation of theory and models into comparative effectiveness research • Role of simulation