Infusion of Statistical Science Into Comparative Effectiveness Research: What, Why, How, Who? Sally C. Morton Statistics

1. Infusion of Statistical Science Into Comparative Effectiveness Research:What, Why, How, Who?Sally C. MortonStatistics and Epidemiology, RTI International

2. What is Comparative Effectiveness Research (CER)? “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.” Institute of Medicine Committee on Comparative Effectiveness Research Prioritization (2009)

3. Other Names for CER: “Clinical Effectiveness Research” or “Patient-Centered Research” “Which treatment works best, for whom, and under what circumstances?” (Slutsky and Clancy. Am J Med Qual 2009;24:67-70) • Informs a clinical decision (screening, diagnosis, treatment) • Focuses on effectiveness not efficacy • Compares at least 2 alternatives, one may be “usual care” • Measures outcomes that matter to patients, including harms and benefits, at both the population and subgroup levels • Includes generation and analysis of new evidence; secondary data analysis; and synthesis Institute of Medicine Committee on Comparative Effectiveness Research Prioritization (2009)

4. Why CER Now? • Focus on healthcare reform • Clinical practice often not based on evidence • Coverage decisions often unrelated to effectiveness • Large and escalating volume of information • Synthesis required • Relevance to clinical practice unclear • CER expected to increase quality, effectiveness, and efficiency by • Summarizing existing knowledge • Identifying knowledge gaps

5. It’s Where the Money Is Dramatic increases in funding at AHRQ for CER • $ 30 million in FY07 • $ 50 million in FY08, now $88 million ARRA (Stimulus Bill) targeted for CER • $ 1.1 billion in FY09 and FY10 • $300 million to AHRQ • $400 million to NIH • $400 million to HHS Office of the Secretary • Institute of Medicine (IOM) Committee to recommend priorities for CER

6. How Do Statistical Methods Contribute To Comparative Effectiveness Research? Evidence generation: prospective and restrospective analysis of secondary databases such as medical registries and claims data Evidence generation: primary data collection via clinical trials and observational studies Evidence synthesis: systematic reviews and meta-analyses Prioritization of gaps in evidence generation, synthesis, communication, or translation Evidence translation: generation of information for different decision-makers, e.g., clinical practice guidelines Evidence synthesis: cost-effectiveness analysis and decision modeling Evidence communication: dissemination of findings Challenge: Statistical methods that are scientifically rigorous, relevant (patient-centered), and timely

7. Factors That Enter Into Clinical Decisions Mulrow et. al. Ann Intern Med 1997;126:389-391

8. Continuum From Research Studies to Clinical Practice Guidelines

9. Focus on Evidence Generation Via Primary Data Collection Studies • RCTS were recommended for 49 of the top 100 CER research priorities identified by the IOM • “The paradox of the clinical trial is that it is the best way to assess whether an intervention works, but arguably the worst way to assess who will benefit from it.” (Mant. Lancet 1999;353:743-46) • Knowing a treatment is efficacious is necessary but not sufficient for knowing it is effective

10. Distinguishing Efficacy and Effectiveness

11. Distinguishing Efficacy and Effectiveness(Cont)

12. How Do We Expand CER Beyond Explanatory (Ideal Setting) RCTs How can we improve clinical trial organizational efficiency, analytic efficiency, and usability for decision-makers? (Luce et al. Ann Intern Med 2009;151:206-209) • Pragmatic trials (real-world settings) • Cluster randomized trials • Bayesian adaptive designs How can we improve methods to control for confounding in observational studies? • Propensity scores • Instrumental variables

13. Focus on Evidence Synthesis: Unadjusted Indirect Comparison A P A P A P A P A D A D A E A F B P B P B P B C B C B D B E B F How does one compare A vs. B when no direct (head to head) evidence is available?

14. Adjusted Indirect Comparison A P A P A P A P A D A D A E A F B P B P B P B C B C B D B E B F In this approach, the comparison is adjusted by a common comparison group Alternative approach: network analysis

15. Overarching Principle: “Fit For Purpose” “Decisions about the use of therapeutic interventions, whether for individuals or entire healthcare systems, should be based on the totality of available evidence. The notion that evidence can be reliably or usefully placed in ‘hierachies’ is illusory. Rather, decision makers need to exercise judgement about whether (and when) evidence gathered from experimental or observational sources is fit for purpose.” Rawlins. Clin Med 2008;8:579-88.

16. Questions to Ponder • How can RCTs be designed to most benefit CER? • When and how can observational studies complement experimental studies to answer CER questions? RCTs ≡ efficacy Observational Studies ≡ effectiveness

17. Questions to Ponder • What methods for observational studies adequately address confounding? • What meta-analytic methods for indirect evidence will inform CER questions? • Whose data are they? • What are the statistical methods research priorities? Priority list: A product of this conference

18. Comparative Effectiveness Research (Sort of) new wine Emphasis is driven by technology availability, payer interest, rising chronic disease burden New bottle Federal and payer interest high in the next few years Critical to be successful early I encourage the statistical community to have a glass! Adapted from lecture by Tim Carey on CER