Conference on Statistical Issues in Clinical Trials: From Bench to Bedside to Community

1. Conference on Statistical Issues in Clinical Trials: From Bench to Bedside to Community Bryan R. Luce, Ph.D., MBA University of Pennsylvania April 27, 2010

2. To Meet the CER Challenge We Need Inspirational & Transformational Change • Investment (Private & Public) • Policy • Research Methods • Thresholds for Decision-Making

3. In My Few Minutes, I Will Focus On… Investment (Private & Public) Policy Research Methods Thresholds for Decision-Making 3

4. Inspirational & Transformational Change in Research Methods Example in Trial Methods for CER

5. Traditional Comparative Trials are • Costly • ALLHAT* ($135 M)) • CATIE** ($40 M • Take lots of time • ALLHAT* (~ 8 yrs) • CATIE** (~ 4 yrs) • Inflexible (clinical practice & innovations can change faster than the research (e.g. ALLHAT) and can be… *Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack **Clin Antipsychotic Trials of Intervention Effectiveness

6. Risky Ventures for Manufacturers, e.g…. • PROVE IT*** • ENHANCE**** Neither trial concluded in favor of the sponsor’s product. *** Pravastatin or Atorvastatin Evaluation & Infection Therapy–Thrombolysis in MI ****Effect of Combination Ezetimibe & High-Dose Simvastatin vs. Simvastatin Alone on the Atherosclerotic Process in Patients with Heterozygous Familial Hypercholesterolemia

7. So, to Answer Bob Temple’s Question as to Why Manufacturers Don’t Fund Comparative Trials* • Cost • Time • Risk *Tunis, Strayer, Clancy, JAMA 2003

8. Nevertheless, we see… • Major new federal funding for CER trials and highly likely to see… • More “conditional coverage” policies by payers • E.g Medicare’s CED Policy • Increased interest in CER trials by manufacturers to preempt the above However…

9. …to my mind… CER trials are unsustainable without transformational change in design to generate useful evidence more efficiently

10. For Example…. • CER “problem” • Has “learn & confirm” feel to it • Seeks to add evidence to existing evidence base (e.g., from a systematic review of clinical literature) • Payers & docs want: • To predict how new HC products & services “fit” into local clinical settings & patient groups • Feedback to “learn” what works, for whom, under what conditions • Conditional coverage (CMS’ CED) policy can be thought of as a “learn & firm-up” coverage policy .

11. This is akin to the.. IOM EBM Roundtable “Learning Health Care System” concept ….all of these are …essentially, Bayesian and/or adaptive concepts dealing with Bayesian expressed problems.

12. Problem is fundamentally Bayesian in Nature • By the time we get to the “real world”, there commonly already is evidence (in “Bayese”, we have a “prior”) • And often the “prior” is substantial (but not sufficient) • For policy (e.g. coverage) decisions, we may only need “enough” targeted information to “tip” a decision • Problem often isn’t yes or no, but for whom, when, under what conditions, thus requiring flexibility in evidence development. • Needs of decision makers are best described in probability terms, a specialty of Bayes techniques

13. The Ideal CER Trial/Learning Process • Builds on what is known • Asks questions decision makers want answered • Chooses relevant competitors • Addresses patient heterogeneity • Learns (and adapts) as it accumulates knowledge • Stops when “just enough” evidence is generated for an informed decision.

14. We Also Need Inspirational & Transformational Change in Thresholds for Decision Making I would argue that thresholds relative to tolerance of uncertainty varies by the decision and the decision maker

15. Threshold Standards for Acceptance

16. The Arbiter of the Threshold Should be the Relevant Decision Maker • Safety & efficacy for registration: FDA • Clinical guidelines: Medical specialty societies • Coverage, Reimbursement, Pricing: Health plan, employer, patient advocacy groups • Individual physician-patient decisions: Doc & patient

17. Thus, “Confidence” Levels Would Vary, e.g. • Safety & efficacy for registration: ~95% • CER for Clinical guidelines: ~80% • Coverage, Reimbursement, Pricing: ~70% • Individual physician-patient decisions: ~ 51+%

18. President Kennedy Once Said of General Curtis LeMay “If we go to war with the Soviet Union, I would want General LeMay in the lead plane….

19. “I just wouldn’t want him to make the decision whether to go.” President John F. Kennedy, circa 1962

20. Similarly, for CER, Once the Decision is Made to Design a CER Trial….. I would want the statistician to design and analyze it…

21. I just wouldn’t want her to set the statistical (or probability) threshold as to accepting it.

22. In Sum: Issues in Clinical Trials: From Bench to Bedside to Community (i.e. CER) More important than increasing precision, reducing bias in study design (although admittedly both are important) is “Transformational Thinking”

23. In terms of developing useful CER evidence to inform decision-making, I opt for… • “Effectiveness” over “efficacy” • “Accuracy” over “precision” With respect to the primacy of effectiveness over efficacy…

24. …According to Mishan (1972) “…an imprecise estimate of the right concept is superior to a precise estimate of a wrong concept” And in 1988, Read put it another way…

25. “It is better to be approximately right than precisely wrong.” (L. Read, 1988) Read goes on to graphically depict his notion of accuracy being superior to precision.

26. The value of accuracy over precision Source: Reed JL. From Medical to Socioeconomic Evaluation of Drug Therapy. p.79. In: van Eimeren W, Horisberger B (eds). Socioeconomic Evaluation of Drug Therapy. Springer-Verlag, Berlin: 1988.

27. Thank You!