Advanced Statistical Methods for Translational Research

1. Advanced Statistical Methodsfor Translational Research Context, Process, Purpose

2. My employer provides testing and consulting services to medical device companies Multiple Disclosure Statement of Financial Interest Within the past 12 months, I or my spouse/partner have had a financial interest/arrangement or affiliation with the organization(s) listed below. Affiliation/Financial Relationship Company All TCT 2018 faculty disclosures are listed online and on the App.

3. We are driven to make a scientific contribution to every medical device in the world

4. Why do we need “advanced” statistical methods? Better answers Faster answers Cheaper answers

5. Dangers of “advanced” statistical methods? Wrong answers Delayed answers Expensive answers

6. Dangers of “advanced” statistical methods? Wrong answers Delayed answers Expensive answers “Being a statistician is like being in Alcoholics Anonymous. You try as hard as you can to not fool yourself” Paraphrasing Jim Hodges

7. Examples of advanced methods Multiple imputation for missing data Propensity score adjustment for non-randomized comparisons Finkelstein-Schoenfeld / “Win-ratio” for composite endpoints Hiearchical models Bayesian approaches: sample size re-estimation, using historical data

8. How do we evaluate a new statistical method? • Does the novel method address a practical problem? • What problems does a novel method raise? • Role of pre-specification • Operating characteristics • Urgency of public health need ,

9. Does the method address a practical problem? • If not, why not use a traditional / non-advanced method? • How serious is the practical problem? • How well does the method address it?

10. What problems does a novel method raise? • Are the assumptions of the method reasonable at the design stage? • Are they met for the data we obtained? • How thoroughly has the new method been evaluated? • Has it been evaluated for the specific circumstances at hand?

11. Role of pre-specification • When we see a new method applied, was it pre-specified? • If not, consider the use of the method an exploratory case-study • If it was pre-specified, was the study design aligned? • Sample size calculations should be based on the planned analysis method • Crossover studies, longitudinal studies, etc., require special methods

12. Operating characteristics • Higher power (low type II error rate) • Low chance of false-positive (low type I error rate) • Under what conditions is the new method optimal/non-optimal?

13. Urgent public health need • Sometimes we need faster answers • HIV/AIDS crisis, Ebola vaccine, opioid crisis • Device improvements to solve urgent device-related problems • Need for high power is the driving consideration • What tradeoffs of other considerations can be made? • Considerations not purely statistical, e.g. shorter term endpoints

14. P-values… “Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold” From: The American Statistical Association Statement on P-values Context, Process, and Purpose

15. A mock example • We have a study with P<0.05!!! • Based on a small study • Clinically meaningful effect size • Analysis based on KM analysis • KM stands for Kushner-Manafort test • Just kidding – it’s Kaplan-Meier/log-rank test, but the test was not pre-specified

16. A more realistic example:Kaplan-Meier analysis was once new Addressed practical problems • Better handling of time to event data • Better handling of censored data (losses-to-follow-up) New problems? • Difficulty interpreting? This improved with usage • Glosses over issue of informative censoring

17. Another example:Evaluating methods for “borrowing data” • Bayesian methods can be used to “borrow” strength from prior work • Formalizes incorporation of existing knowledge • Posterior = current data x prior data How much strength should we borrow?

18. Hierarchical Models

19. Hierarchical Models • Reasonable operating characteristics • Only applicable when we have a hierarchical structure • Somewhat challenging to describe how much we’re “borrowing”

20. Bayesian power prior Traditional Bayesian inference: Nfinal= Ncurrent+ Nprior Power prior inference Nfinal= Ncurrent+ a*Nprior Prior counts as much as current data

21. Bayesian power prior Traditional Bayesian inference: Nfinal= Ncurrent+ Nprior Power prior inference Nfinal= Ncurrent+ a*Nprior For a <1, prior has less weight

22. How should we weight historical data? Historical and current data similar (Easy case) Historical and current data different Weight equally? Ignore prior? Posterior Historical data Current data

23. Can the weight be dynamic? adaptive? Allow weight to vary as a function of agreement? More agreement = more weight Less agreement = less weight

24. Advantages of dynamic prior Better type I error rate Less bias

25. When should we prefer fixed vs. dynamic borrowing? While both approaches rely on theory and data… Fixed borrowing can be considered theorydriven We believe a priori that historical and current data should be similar Dynamic borrowing can be considered data driven We allow that historical and current data may or may not be similar

26. When should we prefer fixed vs. dynamic borrowing?

27. Criticisms of dynamic borrowing and responses http://mdic.org/cts/vp/ MDIC Workshop on Virtual Patient Methodology

28. For advanced statistical methods, consider… Not just the p-value but the Context, Process, and Purpose

29. Thank you!