Outline

1. Pragmatic Clinical TrialsNaihua Duan, Ph.D.Professor of Biostatistics (in Psychiatry) Director, Division of Biostatistics and Data Coordination Departments of Psychiatry and Biostatistics, Columbia Universityand New York State Psychiatric Institute Pragmatic clinical trials

2. Outline • Explanatory trials vs. pragmatic trials • Comparative effectiveness research • Randomized encouragement trial • Risk-based allocation design • Sequential multiple assignment randomization trial (SMART) Pragmatic clinical trials

3. Explanatory vs. Pragmatic Trials (I) • Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967 Aug;20(8):637-48. • Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003 Sep 24;290(12):1624-32. Pragmatic clinical trials

4. Pragmatic clinical trials

5. Pragmatic clinical trials

6. Pragmatic clinical trials

7. Explanatory vs. Pragmatic Trials (II) • Explanatory: lab condition, control for contextual factors – “everything being equal…” • Pragmatic/practical: naturalistic condition, incorporate contextual factors – treatment bundle • Medication vs. therapy: attention “bias” • Distinct research questions, neither right or wrong • Efficacy vs. effectiveness Pragmatic clinical trials

8. Explanatory vs. Pragmatic Trials (III) • Explanatory perspective dominated clinical research • FDA largely takes the explanatory perspective • Pragmatic perspective’s important role begin to be recognized in recent years, especially after Tunis, Stryers, and Clancy (2003) • The emergence of comparative effectiveness research as a central component of Obama administration’s health care reform gives prominence to the pragmatic perspective Pragmatic clinical trials

9. Comparative Effectiveness Research (I) Institute of Medicine’s definition: Comparative effectiveness research (CER) is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels. Pragmatic clinical trials

10. Comparative Effectiveness Research (II) • Focus on clinical decisions • Pragmatic clinical trials are natural for CER • Which therapeutic choice works best for whom, when, and in what circumstances? • Kravitz R, Duan N, Braslow JT. Evidence Based Medicine, Heterogeneity of Treatment Effects, and the Trouble with Averages. Milbank Q. Dec 2004;82(4):661–687. • Address the needs of priority populations and sub-groups often under-represented in research • Relevance for Asian population in Hawaii Pragmatic clinical trials

11. Comparative Effectiveness Research (III) • Strongly advocated in Obama’s campaign, strongly supported in Obama administration • US$1.1B budgeted for CER in American Recovery and Reinvestment Act (the “stimulus package) • US$300M for Agency for Health Care and Quality, US$400M for NIH, and US$400M for Secretary of HHS • Long terms funding under consideration Pragmatic clinical trials

12. Comparative Effectiveness Research (IV) • ARRA establishes Federal Coordinating Council for Comparative Effectiveness Research • ARRA also mandates reports from Institute of Medicine and the Federal Coordinating Council by June 30, 2009 to recommend priorities for OS-HHS’s share of CER funds Pragmatic clinical trials

13. Recent Reports on CER • Committee on Comparative Effectiveness Research Prioritization, Institute of Medicine. Initial National Priorities for Comparative Effectiveness Research. June 30, 2009, Washington, D.C., National Academies Press. • http://www.nap.edu/catalog.php?record_id=12648 • Federal Coordinating Council for Comparative Effectiveness Research. Report to the President and the Congress on Comparative Effectiveness Research. June 30, 2009, Washington, D.C., USDHHS. • http://www.hhs.gov/recovery/programs/cer/cerannualrpt.pdf Pragmatic clinical trials

14. IOM Report • Recommends 100 priority research topics • E.g., “Establish a prospective registry to compare the effectiveness of treatment strategies for low back pain without neurological deficit or spinal deformity” • 10 Recommendations for a “Robust National CER Enterprise” Pragmatic clinical trials

15. IOM Recommendation #7 The CER Program should devote sufficient resources to research and innovation in the methods of CER, including the development of methodological guidance for CER study design such as the appropriate use of observational data and more informative, practical, and efficient clinical trials Pragmatic clinical trials

16. IOM Recommendation #8 The CER Program should help to develop large-scale, clinical and administrative data networks to facilitate better use of data and more efficient ways to collect new data to inform CER. • The CER Program should ensure that CER researchers and institutions consistently adhere to best practices to protect privacy and maintain security. • The CER Program should support the development of methodologies for linking patient-level data from multiple sources. • The CER Program should encourage data holders to participate in CER and provide incentives for cooperation and maintaining data quality. Pragmatic clinical trials

17. Federal Coordinating Council Report (I) • Federal Coordinating Commission recommends CER data infrastructure as the first priority for OS-HHS’s share of ARRA funding. Also recommends research and human capital infrastructure for CER methodologies, among other objectives. • Calls on an interdisciplinary workforce including biostatistics, epidemiology, mathematics, economics, and ethics to work on CER. Pragmatic clinical trials

18. Federal Coordinating Council Report (II) • Recommends support for “the development of methods for linking and using databases for CER, the development of new methodologies for pragmatic trials, effective translation and adoption of CER findings into practice, modeling approaches for CER, and evaluation of the impact of CER” • More methods work is needed to advance the state-of-the-art for pragmatic trials and to provide training for using these study designs. Pragmatic clinical trials

19. Outline • Explanatory trials vs. pragmatic trials • Comparative effectiveness research • Randomized encouragement trial • Risk-based allocation design • Sequential multiple assignment randomization trial Pragmatic clinical trials

20. Randomized Encouragement Trial:Partners in Care (PIC) Study • Wells KB, Sherbourne C, Schoenbaum M, Duan N, et al. Impact of disseminating quality improvement programs for depression to primary care: A randomized controlled trial. JAMA. 2000 Jan; 283:212-220. • Schoenbaum M, Unutzer J, McCaffrey D, Duan N, Sherbourne C, Wells KB. The effects of primary care depression treatment on patients’ clinical status and employment. Health Services Research. 2002 October; 37(5): 1145-1158. Pragmatic clinical trials

21. PIC Study Design • Clinic level randomization to QI interventions or usual care • Three arms: UC, QI-Meds emphasizes medication, QI-Therapy emphasizes therapy • Autonomous choice for patient/clinician choice • Study attempts to influence patient/clinician decisions through encouragement, rather than using mandated or blinded assignments as in usual explanatory trials Pragmatic clinical trials

22. Quality Improvement Intervention (I) • Institutional commitment to QI • Training clinicians and nurses • Nurses provide brief clinical assessments, patient education and activation • Nurses follow up with patients to help with medication adherence in QI-Meds • Access to study-trained psychotherapists for cognitive behavioral therapy at a discounted co-pay in QI-Therapy • Patients and providers in all conditions had full choice of treatment, and (in intervention arms) over use of study resources. Pragmatic clinical trials

23. Quality Improvement Intervention (II) • QI intervention includes clinic level and individual level intervention components • Training and care management • Study designed with clinical level randomization • Design effect (variance inflation factor) • Split plot design could be used to assess effect of intervention components separately, and avoid design effect for individual level intervention effects Pragmatic clinical trials

24. QI Trials Are Pragmatic • Not testing efficacy for a specific treatment • Rather, testing effectiveness for a service delivery strategy • Contextual factors incorporated into intervention bundle • Analogous to marketing experiments that aim to increase market share Pragmatic clinical trials

25. PIC Study Sample • 46 clinics randomized • 27322 adult patients screened • 3918 potentially eligible • 1356 enrolled • 1156 followed at month 6 • 1126 followed at month 12 • 805 followed at year 9 Pragmatic clinical trials

26. Impact of QI Intervention • Increase in appropriate care: • At month 6, 51% vs. 40%, p<0.001 • At month 12, 59% vs. 50%, p=0.006 • Decrease in probable depression: • At month 6, 40% vs. 50%, p=0.001 • At month 12, 42% vs. 51%, p=0.005 Pragmatic clinical trials

27. Treatment Effect in QI Trials • Instrumental variables analysis for treatment effect (appropriate care vs. else), using random assignment as the instrumental variable • Reduced rate of probable depression (24% vs. 70%, p=0.001) • Exclusion criterion might be questionable Pragmatic clinical trials

28. Risk-Based Allocation Design (I) • Finkelstein, M.O., Levin, B., & Robbins, H. (1996). Clinical and Prophylactic Trials with Assured New Treatment for Those at Greater Risk. Part I -- Introduction. Amer. J. Public Health. 86:691-695. • Finkelstein, M.O., Levin, B., & Robbins, H. (1996). Clinical and Prophylactic Trials with Assured New Treatment for Those at Greater Risk. Part II -- Examples. Amer. J. Public Health. 86:696-705. • Thistlewaite, D.L., & Campbell, D.T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology. 51:309-317. Pragmatic clinical trials

29. Risk-Based Allocation Design (II) • Non-randomized experiment • Prioritize on baseline risk: treat those with high risk, watch those with low risk *** • Compare those just above threshold (treated) vs. those just below threshold (not treated) • Use those not near threshold to improve the estimation (diagram) • Also known as regression-discontinuity design Pragmatic clinical trials

30. Pragmatic clinical trials

31. Risk-Based Allocation Design (III) • Advantages • Consistent with real-world practice • Facilitates recruitment/retention… • Better external validity than RCT? • More meaningful interpretation of equipoise • Disadvantages • Weaker internal validity than RCT? • Limited sample size near threshold • Challenges in extrapolation towards threshold • Linear extrapolation criticized, but… • Bias-variance trade-off Pragmatic clinical trials

32. Sequential Multiple Assignment Randomization Trial (SMART) • Lavori PW, Dawson R, Rush AJ. Flexible treatment strategies in chronic disease: clinical and research implications. Biol Psychiatry. 2000 Sep 15;48(6):605-14. • Lavori PW, Dawson R. Adaptive treatment strategies in chronic disease. Annu Rev Med. 2008;59:443-53. • Murphy SA. Optimal Dynamic Treatment Regimes (with discussion). Journal of the Royal Statistical Society. 2003;Series B. 65:331-366. • Murphy, SA. An Experimental Design for the Development of Adaptive Treatment Strategies. Statistics in Medicine. 2005;24:1455-1481. Pragmatic clinical trials

33. Why SMART? • Traditional clinical trials (two-armed, treatment vs. control) are too simplistic to be helpful with clinical decision-making in front-line practice settings • Clinical practitioners need to accommodate heterogeneity in patient conditions, patient preferences, and interim outcomes/side effects Pragmatic clinical trials

34. Example of an Adaptive Treatment Strategy Treatment of depression. Goal is to achieve and maintain remission. Provide Citalopram for up to 12 weeks gradually increasing dose as required. If either the maximum dose has been provided for two weeks, or 12 weeks have occurred, yet there is no remission, then if there has been a 50% improvement in symptoms, augment with Mirtazapine. else switch treatment to Bupropion. Else (remission is achieved) maintain on Citalopram and provide web-based disease management. Pragmatic clinical trials

35. SMART • Develop/evaluate adaptive treatment strategies, (ATS) • Identify key decision points • Randomize at each decision point • Evaluation analogous to playing chess or go • Myopic, immediate consequences of move • Long term outcome for successful Game • Backward optimization Pragmatic clinical trials

36. Basic Idea for Constructing the Optimal ATS: Move Backwards Through Stages. Pragmatic clinical trials

37. Discussions • Explanatory trials are essential for treatment development • Pragmatic trials are useful for clinical decision making (and policy decision making?) • Pocock and Elbourne (2000): “Making the trials as much like routine practice as possible may help to make randomized, controlled trials more feasible and their results more widely generalizable.” • Federal Coordinating Commission: “Increased emphasis on well-conducted pragmatic trials could increase acceptance of CER findings.” Pragmatic clinical trials