Advanced Clinical Design and Statistical Analysis Core

1. Advanced Clinical Design and Statistical Analysis Core NIDA P50 Center for Medication Development for Cocaine Dependence

2. The Context: FDA Critical Path Initiative1 The �Pipeline Problem� How do we optimize the rapid progress in basic sciences for developing clinical applications? Two FDA suggestions for improving the efficiency of clinical trials: Bayesian Statistical Analysis Adaptive Trial Design

3. Use of Bayesians Statistical Approaches.2,3 100 current trials at M.D. Anderson have been designed or are being followed using Bayesian approaches. Recently ~10% of FDA medical device approvals are based on Bayesian models. FDA approval of Pravigard Pac (Bristol Myers Squib) based on Bayesian Analyses. FDA was one of the sponsors of the 2004 conference �Can Bayesian Approaches to Studying New Treatments Improve Regulatory Decision Making?� FDA �Document Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials - Draft Guidance for Industry and FDA Staff�4 http://www.statmodel.com/ugexcerpts.shtml SAS has recently released versions of Bayesian procedures for generalized linear models and generalized linear mixed models, including survival analysis.

4. Adaptive Trials Adaption based on a priori specifications.3 Permits specification of a target for the design to optimize. Recent, industry estimate of improved efficiency: ~30% decrease in direct costs for adaptive versus conventional trial designs.5 FDA �Guidance for Industry: Adaptive Design Clinical Trials for Drugs and Biologics - Draft Guidance�6 http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM201790.pdf

5. P50 Advanced Clinical Design and Statistical Analysis Core (ACDSAC) Mission: ACDSAC will focus on increasing the information yield from clinical trials: improving the precision of trial-based estimates to inform decision-making, while optimizing resource use. Methods: Following recommendations by the FDA [1,7,8,9,10,11,12], the ACDSAC will: Implement of Bayesian statistical methods Develop and implement adaptive/flexible trial designs Screen and evaluate bio-behavioral markers Use Bayesian and Frequentist Structural Equation Modeling for evaluating mechanistic hypotheses and bio-behavioral markers.

6. Evaluating Heterogeneity in Pharmacotherapy Trials for Drug Dependence: A Bayesian Approach C. Green, F. G. Moeller, J. Schmitz, J. Lucke, S. Lane, A. Swann, R. Lasky, & J. Carbonari

7. Types of Statistical Reasoning

8. What�s the difference? Bayesians assess which hypotheses are most probable, frequentists evaluate which hypotheses should be rejected, and likelihoodists say which hypotheses are best supported.13

9. Bayesian and Frequentist Statistical Reasoning Similarities Data arises from a random data-generating process. Governed by an unobservable parameter ?. Sample observations to characterize ?. Differences Frequentist Reasoning Probability: For a repeatable event. The limit of the event�s relative frequency occurring in an infinite sequence of similar events. Assumes ? is fixed and unknown Uncertainty in estimates of ? arises from sampling error. Bayesian Reasoning Probability: An event, not necessarily repeatable. Reflects a judgment or degree of belief that the event will occur.14 Acknowledges uncertainty due to sampling error. Defines ? as random and unknown Characterized using a probability distribution.

10. Priors ? Data ? Posteriors Process of �learning�.15 Priors are normative representations of an investigator�s degree of belief.16 Tempering priors permits representation degrees of belief in the scientific community.17 Diffuse/vague, Informative Skeptical, Indifferent, Enthusiastic All the information from the likelihood and the prior are combined in the posterior distribution Summaries of the posterior distribution: Point estimates (e.g. the posterior expectation and variance) Interval estimates (e.g. credible intervals) The posterior distribution of one study can form the prior distribution for the next participant/study. Steve Goodman refers to this as providing a �calculus of the evidence� (p.284)18

11. How Bayesian Analyses Work

12. How Bayesian Analyses Work

13. How Bayesian Analyses Work

14. How Bayesian Analyses Work

15. So What?

16. Benefits of Bayesian Methods3,15 �Learning Theory� Approach Dovetails with the use of Adaptive Trial Designs Incorporation of Prior Information Predictive Distributions Probability Statements serve as a Decision-Making Tool Subgroup Analyses

17. Heterogeneity in Response to Pharmacotherapy for Cocaine Dependence The search for pharmacotherapies for cocaine dependence has not yet produced any medications approved by the FDA.18 Possibly attributable to heterogeneity among cocaine dependent patients.19,20,21,22,23 Gender Ethnicity Employment Status/SES Severity of Dependence Route of Administration Primary Drug Dependence Diagnosis Drug and Alcohol Comorbidities Psychiatric Comorbities Readiness to Change Differences in Dopaminergic Tone Recommendations for characterizing this heterogeneity include the exploratory evaluation of possible subgroups.21,23 May inform the design and analysis of clinical trials.18,20,22,23

18. Subgroup Analysis Controversial in clinical trials .24,25,26,27,28,29,30,31,32,33,34,35. Multiplicity Number of statistical tests ? in conducting sub-group analyses. ? Study-wise Type I Error as number of statistical tests ? Power/Type II Error Type II Errors arise from small sample sizes and small effects ? n in subgroups ? p (Type II Error). Guidelines/recommendations for conduct of subgroup analyses exist.28,30,32,33,34 Problems of elevated Type I and II Error rates persists. Simulation of classical tests of interaction/subgroup effects:28,32 Sample size determination characteristically focuses on main effect of treatment. Optimistic scenario: interaction effect same magnitude as treatment effect N for Power = 80% to detect the main effect Power = 29% to detect the interaction. Interaction effects are typically a fraction of the magnitude of the treatment effect. Bayesian statistical reasoning proposed as a solution.36,37,38,39 Permits statements such as p(subgroup effect of a specified magnitude exists). Avoids the forced dichotomy of statistical hypothesis testing. More effectively evaluate the weight of the existing evidence, than conventional Fisher/Neyman-Pearson (Frequentist) statistical testing.40,41

19. Bayesian Subgroup Analysis Explanation of Bayesian subgroup analysis is provided by Simon.39 Two predictors and their interaction Key parameter is the b-weight associated with the interaction term. Posterior distribution of this parameter relies on the specification of the prior distribution. Simon suggests using skeptical, informative, prior distributions: Centered on the null hypothesis of no effect Low probability of a clinically meaningful effect occurring. Calculations follow from the equation below (p.2911, Equation (1)) 39: Specification: d = meaningful effect size, p = p (d), F = Normal (cdf). Calculation: di = variance of the skeptical prior distribution Current analyses are: Hypothesis generating, not being used to set policy, and not being used to argue for changes in clinical practice. Posit indifferent, diffuse priors for all parameters estimated in the statistical models Sensitivity Analysis Using skeptical priors derived via Simon�s39 approach permits evaluation of the credibility of results based on diffuse priors.

20. Substantive Example A secondary data analysis of a trial of citalopram for treating cocaine dependence.42 How does decision-making, as measured by performance on the Iowa Gambling Task (IGT), moderate the effect of citalopram in reducing longest sustained abstinence measured by consecutive, cocaine-negative urines? 

21. Background Impaired decision-making correlates with substance abuse/cocaine dependence.43,44 Decision-making as measured by the Iowa Gambling Task (IGT) 45 Involves disregard for long-term consequences Related to the non-planning aspects of the broader construct of impulsivity. Impulsivity is a predicts treatment 46,47: Retention Effectiveness Impulsivity is associated with serotonergic (5HT) function. 5HT polymorphisms are associated with differential vulnerability to cocaine contingent reward.48,49,50,51,52,53,54,55 Serotonergic 5-HT2A antagonists and/or 5-HT2C agonists are promising agents for addressing impulsivity with possible resulting effects on cocaine use among cocaine dependent participants.56 Given its role in impulsivity which is implicated in substance use, and its potential responsiveness to serotonergic medications, baseline level of decision-making may moderate the effect of citalopram on cocaine use.

22. Trial Design and Statistical Analysis N = 76; Trial details available elsewhere.42 Powered to detect a main effect of treatment. Not powered to detect an interaction between impulsivity/decision-making and citalopram. Post hoc analysis of the interaction between impulsivity/decision-making and citalopram. Measures Outcome: Longest consecutive number of cocaine-free urines Benzoylcgonine < 300 ng/ml Measured twice weekly Decision-Making Iowa Gambling Task56 Computerized version of the original gambling task Net score of cards selected in each of the five blocks used as an index of decision-making. Statistical Analyses PROC GENMOD (SAS v. 9.2) for analysis of generalized linear models.57 Longest consecutive number of cocaine-free urines was modeled as a Poisson outcome. Evidence for an interaction is examined from two perspectives: A diffuse prior assuming very little regarding the likelihood of the interaction A skeptical prior assuming that an interaction has a low probability of occurring. 

23. Specification of Priors Diffuse Indifferent Priors Log-scale, diffuse prior distributions were specified as ~N(0, 1 x 106). Indicates an initial assumption of no-effect Indicates there is 95% chance that the true log(parameter) falls within a broad range (i.e. �1.96 x or -1960 to 1960). Skeptical, Informative Priors Following Simon22 Parameters except for the interaction term are specified as ~N(0, 1 x 106). Calculation of the prior variance for the skeptical prior assumes: Clinically meaningful effect: Risk-ratio = 1.01. Working in the log-scale: d = log(1.01) = 0.00995. Assuming that this effect is unlikely to occur we set p = 0.025. Skeptical prior centered on no effect: log(risk ratio) = 0) Utilizing the Normal (cdf) yields a variance di = 2.577 x 10-5. Prior for the interaction has the form ~N(0, 2.577 x 10-5) Equipoise regarding the presence of an interaction A clinically meaningful effect is unlikely.

24. Results

25. Posterior Distributions for the Interaction, and Simple Effects using Diffuse Priors

26. Conclusions Methodological concerns complicate sub-group analysis. Requisite quantitative strategies must effectively weigh the evidence regarding heterogeneity. Heterogeneity detected in Bayesian subgroup analysis may be incorporated into future trial designs via stratified or urn randomization methods. Bayesian methods better serve this function than Frequentist approaches: Permits statements regarding the probability that an interaction exists. Consideration of the empirical evidence from various perspectives to evaluate sensitivity to prior assumptions. Systematic revision of prior distributions into posterior distributions based on the observed data: Provides a bridge across studies Informative, empirically defensible, priors resulting from current trials may lead to smaller more efficient trials. Provides a mechanism for data analysis required in adaptive designs.

27. New Direction Potential combination of studies as informative priors for analysis of subgroup effects. Power Priors58: Permits evaluation of degree to which combination is beneficial. Increased precision around point estimates.

28. Declaration of Interest and Acknowledgements Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. AcknowledgementsThis study was funded by grants from the National Institute on Drug Abuse (5R01DA008425 and P50-DA-9262) and the University of Texas Medical School at Houston (Clinical Investigator Award). Preliminary results were presented at the annual meeting of the College on Problems of Drug Dependence in 2008. Results are currently in press: Green, C., Moeller, F., Schmitz, J., Lucke, J., Lane, S., Swann, A., Lasky, R. & Carbonari, J. (2009). Evaluation of Heterogeneity in Pharmacotherapy Trials for Drug Dependence: A Bayesian Approach. The American Journal of Drug and Alcohol Abuse. 

29. References FDA. (March, 2004) Innovation or Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products. Berry, D.A. (2006) Bayesian clinical trials. Nature Reviews: Drug Discovery. 5(1):27-36. Berry, D. (2008) Bayesian Biostatistics. Bayesian Biostatistics Conference. Houston, TX. January 30, 2008. FDA. (2006) Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials � Draft Guidance for Industry and FDA Staff. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health, Division of Biostatistics, Office of Surveillance and Biometrics. Dragalin, V., Burns, D., Fardipur, P. & Krams, M. Current use of Bayesian Methods in Clinical Research. Bayesian Biostatistics Conference. University of Texas M.D. Anderson Cancer Center. Houston, Tx. 2008. FDA �Guidance for Industry: Adaptive Design Clinical Trials for Drugs and Biologics - Draft Guidance� U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research CBER. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM201790.pdf Gottlieb, S. Speech before 2006 Conference on Adaptive Trial Design. Washington, D.C. July, 2006. http://www.fda.gov/oc/speeches/2006/trialdesign0710.html. Hung, H.M.J., O�Neill, R.T., Wang, S.,-J., & Lawrence, J. A Regulatory View on Adaptive/Flexible Clinical Trial Design. Biometrical Journal. 48(4): 565-573. 2006. O�Neill, R.T. FDA�s Critical Path Initiative: A Perspective on the Contribution of Biostatistics. Biometrical Journal. 48(4): 559-564. 2006. Woodcock, J. FDA introductory comments: clinical studies design and evaluation issues. Clinical Trials. 2: 273-275. Temple, R. How the FDA currently makes decisions on clinical studies. Clinical Trials. 2: 276-281. Irony, T. Bayesian Medical Device Clinical Trials in the Regulatory Setting. Bayesian Biostatistics Conference. University of Texas M.D. Anderson Cancer Center. Houston, Tx. 2008. Sober, Elliott. Instrumentalism, Parsimony, and the Akaike Framework Source: Philosophy of Science, Vol. 69, No. 3, Supplement: Proceedings of the 2000 Biennial Meeting of the Philosophy of Science Association. Part II: Symposia Papers (Sep., 2002), pp. S112-S123. Howson, C. & Urbach, P. (2006), Scientific reasoning: The Bayesian approach, Open Court, Chicago; Schervish, M. J. (1995), Theory of statistics, Springer-Verlag, New York, NY Spiegelhalter, D.J., Abrams, K.R. & Myles, J.P. (2004) Bayesian Approaches to Clinical Trials and Health-Care Evaluation. John Wiley & Sons Ltd. West Sussex, England. Hacking, I. (2001), An introduction to probability and inductive logic, Cambridge University Press, Cambridge Shimony, A. (1993), �Scientific inference� In Search for a naturalistic world view: Scientific method and epistemology, Cambridge University Press, Cambridge, UK. Goodman S. M. Introduction to Bayesian Methods I: measuring the strength of evidence. Clin Trials. 2005;2(4):282-90.

30. References 18. de Lima, M.S., de Oliveira Soares, B.G., Pereira Reisser, A. A., & Farell, M. (2002). Pharmacological treatment of cocaine dependence: a systematic review. Addiction. 97, 931-949. 19. de Lima, M.S., de Oliveira Soares, B.G., Pereira Reisser, A. A., & Farell, M. (2002). Pharmacological treatment of cocaine dependence: a systematic review. Addiction. 97, 931-949. 20. Elkashev, A. & Vocci, F.(2003). Biological markers of cocaine addiction: implications for medications development. Addiction Biology. 8, 123-139. 21. Carroll, K. M., Nich, C. & Rounsaville, B.J. (1997). Variability in treatment-seeking cocaine abusers: implications for clinical pharmacotherapy trials. NIDA Research Monographs. 175: 137-157. 22. Nunes, E. V. (1997). Methodologic recommendations for cocaine abuse clinical trials: a clinician-researcher�s perspective. NIDA Research Monographs. 175: 73-95. 23. Elkashef, A., Holmes, T.H., Bloch, D.A., Shoptaw, S., Kampman, K., Reid, M.S., Somoza, E., Ciraulo, D., Rotrosen, J., Leiderman, D., Montgomery, A., Vocci, F. (2005) Retrospective analyses of pooled data from CREST I and CREST II trials for treatment of cocaine dependence. Addiction. 100 (Supplement. 1) 91-101. 24. Hernandez, A.V., Boersma, E., Murray, G.D., Habbema, J.D.F., & Steyerberg, E. W. (2006). Subgroup analyses in therapeutic cardiovascular clinical trials: Are most of them misleading? American Heart Journal. 151: 257-264). 25. Counsell, C. E., Clarke, M.J., Slattery, J. & Sandercock, P. A. G. (1994). The miracle of DICE therapy for acute stroke: fact or fictional product of subgroup analysis? British Medical Journal. 309: 1677-1681. 26. ISIS-2 Collaborative Group Randomized trial of IV streptokinase, oral aspirin, both, or neither among 17187 cases of suspected myocardial infarction. (1988). Lancet. 2: 349-360. 27. Feinstein, A. R. (1998). The Problem of Cogent Subgroups: A Clinicostatistical Tragedy. Journal Clinical Epidemiology. 51(4): 297-299. 28. Brookes, S.T., Whitely, E., Egger, M., Smith, G.D., Mulheran, P.A. & Peters, T.J. (2001). Subgroup Analyses in randomized controlled trials: quantifying the risks of false-positives and false negatives. Executive Summary. Health Technology Assessment. 5(33). 29. Assmann, S.F., Pocock, S.J., Enos, L.E. & Kasten, L.E. (2000). Subgroup analysis and other (mis)uses of baseline data in clinical trials. The Lancet. 355: 1064-1069. 30. Pocock, S.J., Assmann, S.E., Enos, L.E. & Kasten, L.E. (2002). Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Statistics in Medicine. 21: 2917-2930. 31. Rothwell, P.M. (2005). Treating Individuals 2: Subgroup analysis in randomized controlled trials: importance, indications, and interpretations. The Lancet. 365: 176-186. 32. Brookes, S.T., Whitely, E., Egger, M., Smith, G.D., Mulheran, P.A. & Peters, T.J. (2004). Subgroup Analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for interaction tests. Journal of Clinical Epidemiology. 57 (229-236). 33. Adams, K.F. (1998). Post hoc subgroup analysis and the truth of a clinical trial. American Heart Journal. 136(5): 753-758. 34. Cook, D. I., Gebski, V.J. & Keech, A.C. (2004) Subgroup analysis in clinical trials. Medical Journal of Australia. 180: 289-291. 35. Guillemin, F. (2007). Primer: the fallacy of subgroup analysis. Nature Clinical Practice: Rheumatology. 3(7): 407-413. 36. Dixon, D.O. & Simon, R. (1991) Bayesian Subset An�lysis. Biometrics. 47: 871-881. 37. Simon, R. & Freedman, L.S. (1997). Bayesian Design and Analysis of Two X Two Factorial Clinical Trials. Biometrics. 53: 456-464. 38. Simon, R., Dixon, D.O. & Freidlin, B. (1996). Bayesian Subset Analysis of a Clinical Trial for the Treatment of HIV Infections. In Bayesian Biostatistics (Statistics: a Series of Textbooks and Monographs). Eds. Donald Berry and Dalene Stangl. Marcel Dekker, New York. 39. Simon, R. (2002) Bayesian Subset Analysis: Application to studying treatment-by-gender interactions. Statistics in Medicine. 21: 2909-2916. 40. Goodman, S.N. (1999). Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy. Annals of Internal Medicine. 130:995-1004. 41. Johnstone, D.J. (1986). Tests of significance in theory and practice. The Statistician. 35:491-504.

31. References 42. Moeller, F.G., Schmitz, J.M., Steinberg, J.L., Green, C.M., Reist, C., Lai, L.Y., Swann, A.C., Grabowski, J. (2007). Citalopram combined with behavioral therapy reduces cocaine use: A double-blind placebo controlled trial. American Journal of Drug and Alcohol Abuse.33(3): 367-378. 43. Bechara, A. & Damasio, H. (2002). Decision-making and addiction (part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences. Neuropsychologia. 40: 1675-1689. 44. Stout, J.C., Busemeyer, J. R., Lin, A., Grant, S.J. & Bonson, K.R. (2004). Cognitive modeling analysis of decision-making processes in cocaine abusers. Psychonomic Bulletin and Review. 11(4), 742-747. 45. Moeller FG, Barratt ES, Dougherty DM, Schmitz JM and Swann AC (2001) Psychiatric aspects of impulsivity. Am J Psychiatry 158:1783-1793. 46. Coffey SF, Gudleski GD, Saladin ME and Brady KT (2003) Impulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals. Exp Clin Psychopharmacol 11:18-25. 47. Patkar AA, Murray HW, Mannelli P, Gottheil E, Weinstein SP and Vergare MJ (2004) Pre-treatment measures of impulsivity, aggression and sensation seeking are associated with treatment outcome for African-American cocaine dependent patients. J Addict D/s 23:109-122. 48. Peroutka SJ (1998) Serotonin receptor variants in disease: New therapeutic opportunities? Ann NY Acad Sci 861:16-25. 49. Gothert M, Propping P, Bonisch H, Bruss M and Nothen MM (1998) Genetic Variation in Human 5-HT Receptors: Potential Pathogenetic and Pharmacological Role. Ann NY Acad Sci 861:26-30. 50. Cravchik A and Goldman D (2000) Neurochemical individuality: genetic diversity among human dopamine and serotonin receptors and transporters. Arch Gen Psychiatry 57:1105-1114. 51. Bjork JM, Moeller FG, Dougherty DM, Swann AC, Machado MA and Hanis CL (2002) Serotonin 2a receptor T102C polymorphism and impaired impulse control. Am J Med Genet 14:336-339. 52. Glatt CE, Tampilic M, Christie C, DeYoung J and Freimer NB (2004) Re-screening serotonin receptors for genetic variants identifies population and molecular genetic complexity. Am J Med Genet B Neuropsychiatr Genet 124:92-100. 53. Patkar AA, Mannelli P, Peindl K, Hill KP, Gopalakrishnan R and Berrettini WH (2006) Relationship of disinhibition and aggression to blunted prolactin response to meta-chlorophenylpiperazine in cocaine-dependent patients. Psychopharmacology (Be/t)1-10. 54. Mannelli P, Patkar AA, Peindl K, Tharwani H, Gopalakrishnan R, Hill KP and Berrettini WH (2006) Polymorphism in the serotonin transporter gene and moderators of prolactin response to meta-chlorophenylpiperazine in African-American cocaine abusers and controls. Psychiatry Res 144:99-108. 55. Kreek MJ, Bart G, Lilly C, LaForge KS and Nielsen DA (2005) Pharmacogenetics and human molecular genetics of opiate and cocaine addictions and their treatments. Pharmacol Rev 57:1-26. 56. Bechara A, Damasio H, Tranel D and Damasio AR (1997) Deciding advantageously before knowing the advantageous strategy. Science 275:1293-1295. 57. SAS Institute Inc. (2006). Preliminary Capabilities for Bayesian Analysis in SAS/STAT� Software. Cary, NC: SAS Institute Inc. 58. Ibrahim, J.G. & Chen, M.H. (2000). Power Prior Distributions for Regression Models. Statistical Science, 15, 46-60.