Laura Schmidt, Professor

1. Health Care Systems EPI 247: Week 10Framing Aims and Hypotheses in Implementation Research (a.k.a., how to use what you’ve learned in this class) Laura Schmidt, Professor

2. Applying What You’ve Learned in this Class:The difference between common sense and science: The underlying logic or conceptual framework is EXPLICIT and TESTED.

3. Common Sense • Implicit assumptions about the way things work • Ad hoc approach to analysis • Categories and ideas arise from personal life experience – “hubris” • Sometimes accurate for situations one knows well • Hard to subject to serious scrutiny and correct errors because underlying assumptions are not explicit • Often masquerades as “descriptive” and “clinically informed” case-based research

4. Conceptual Framework

5. Without a Conceptual Framework, It Isn’t Science • You are always operating with a theory whether you think so or not • If the theory isn’t explicit, then you may be operating on the basis of common sense • Common sense, based on life experience, translates into an underlying bias

6. What Theory Does • Simplifies a complex problem: focuses analysis on a smaller set of critical relationships • Organizes the analysis by defining the categories in which phenomena are to be grouped • Allows one to imagine testable hypotheses • Creates linkages to scientific and policy communities by allowing for replication, generalized knowledge and dialogue about broader values and interests

7. Suggested mantra: “Be clear enough to be proven wrong”

8. MODEL CONCEPTUAL FRAMEWORK THEORY Levels of Analytic Framing

9. THEORY Charles Darwin British Naturalist 1809 -1882 “I have called this principle, by whicheach slight variation, if useful, is preserved, by the term Natural Selection.”  —Charles Darwin from "The Origin of Species" CHARACTERISTICS: Simplifies a complex problem High level of abstraction allows for broad applicability Concepts and assumptions are EXPLICIT, even TRANSPARENT Theory may be ultimately proven wrong, but it is testable Has a broad vision of history, human behavior, values

10. What is an Hypothesis? • “A tentative assumption made in order to draw out and test its logical empirical consequences.” • Your “best bet” among all alternatives • It must be empirically testable (by your study) • You must have an alternative strategy, should the hypothesis prove invalid, that you present later. • By testing the hypothesis, you must be able to attain the proposal’s objective

11. AN EXAMPLE • Long-term goal • To advance our understanding of how to effectively and efficiently implement much-needed reforms in safety net healthcare systems. • Overall objective: • To identify specific barriers to implementing the primary care reforms in the Greater New Orleans safety net, and how successful medical homes overcome those barriers. • Central Hypothesis: • Barriers to implementation—and strategies for overcoming those barriers—will vary across types of practices (e.g., FQHCs, storefronts) and the 7 elements of the PCMH model. (

12. Aim 1: Descriptive • Specific Aim 1: To validate implementation of the PCMH model within each of five NCQA-recognized primary care practices over three years using quantitative data on process changes within each practice, and trends in patient outcome and costs. • Hypothesis 1a. At the practice level, different types of safety net practices (e.g., FQHC, faith-based, governmental) will vary in the kinds of process changes they have implemented to achieve NCQA recognition (e.g., improving information technology, outreach, continuity of care, efficiency). • Hypothesis 1b. At the patient level within each of the five practices, PCMH-based process changes will be associated with a trend towards improvement in patient outcomes (e.g., blood pressure control) over a three-year period. • Hypothesis 1c. At the patient level within each of the five practices, we will further explore the extent and nature of change in the costs of care over the three-year study period.

13. Aim 2: Explanatory • Specific Aim 2: To better understand barriers to implementing the PCMH model, particularly in the healthcare safety net, and how successful practices overcome those barriers, through qualitative case studies. • Research Question 2a. Do different types of primary care practices in the safety net (e.g., FQHC, university-affiliated) face different barriers to implementing the PCMH model (e.g., barriers related to funding, technical expertise, clinician adherence)? • Research Question 2b. Where diverse practices face common barriers to implementing the medical homes model, do they come up with the same or different strategies for overcoming those barriers? • Research Question 2c. Are there particular strategies for overcoming implementation barriers that, across practices, seem more effective than others? Research Question • 2d. Across successful practices, is there an optimal timing and sequence for rolling out different elements of the PCMH model (e.g., information technology infrastructure development, followed by systems to improve management of chronic illness)? • Research Question 2e. What is the impact of PCMH transformation on patient and provider experience and satisfaction?

14. Common Pitfalls in Conceptual Framing OVERSHOOTING: - theory too broad for the empirical problem - too much theory (KISS) - too much jargon and “insider dialogue” - over-interpreting results* UNDERSHOOTING: - failure to make analytic assumptions explicit - re-inventing the wheel - failure to take on what’s really important

15. Don’t Over-Interpret Results STRONG INFERENCE REQUIRES ROBUST RESULTS: -accumulation of evidence -same relationships observed across very different cases -theory or hypothesis has been tested against plausible competing hypotheses

16. Some Tricks of the Trade THINK COMPARATIVELY -Build in similar and contrast cases -Select cases to hold some variables constant while systematically varying others of interest -Exploit highly different cases to make broader generalizations, stronger inferences and to reveal limits of the theory’s applicability

17. Many Sources for Comparative Work • PEOPLE: cases and controls, different stakeholder groups • TIME: before/after policy comparisons, implementation process • PLACE: same policy in two political contexts • IDEAS: different policy agendas in the same place • OUTCOMES: failures versus success stories • THEORIES: competing theories applied to the same case

18. Other Tricks of the Trade • Don’t sample on the dependent variable • Avoid mismatches between theory, study design, and data (e.g.,unit of analysis) • Explore your own biases and assumptions • Apply 3 theories to the same problem • Write the paper’s introduction before looking at the data • Then move between theory and data iteratively • Above all, READ THE LITERATURE

19. That’s All, Folks