All Is Not Well in the World of Translational Research

1. All Is Not Well in the World of Translational Research Ellis F. Unger, M.D. Office of Drug Evaluation-I Office of New Drugs Center for Drug Evaluation and Research U.S. Food and Drug Administration June 21, 2012

2. All Is Not Well in the World of Translational Research Disclaimer: The views expressed herein are mine, and do not necessarily represent the views/position of the U.S. Food and Drug Administration.

3. 1985 – 1997: • Translational research, Cardiology Branch, NHLBI, NIH • 1988 – 1997: • Member (chair), Animal Care and Use Committee, NHLBI, NIH • 1995 – 1997: • Clinical Investigator (IND Sponsor), NHLBI, NIH • 1997 – present: • Medical Officer (CBER)  (acting) Office Director (CDER), FDA My Interest in the Topic:

4. What’s the Problem with Translational Research? • Bias • Lack of knowledge • Lack of oversight • Lack of rigor • Lack of standards • Standards for investigators • Standards for journal editors

5. Bias – always present: • Positive results: • Publications • Research grants • Speaking engagements • Consulting arrangements • Patents; licensing arrangements • Career advancement

6. Bias – always present: • Positive results: • Publications • Research grants • Speaking engagements • Consulting arrangements • Patents; licensing arrangements • Career advancement • Negative results: • Mostly wasted time

7. Remediable Limitations of Pre-clinical POF Studies • Protocol may not exist; may exist but be amended without any record or chain of accountability • Studies not universally randomized • Studies not universally blinded • Randomization code and/or blocking strategy may be known (or available) to investigator • Primary endpoint often not identified; most endpoints are exploratory • Just because an endpoint measure is objective, doesn’t mean it isn’t susceptible to bias!

8. Remediable Limitations of Pre-clinical POF Studies • No statistical plan; or • Rudimentary and/or flawed statistical plan: • Statistical test not fit for purpose • Typically no plan to deal with multiplicity (multiple end points, multiple time points, or both) • Typically no plan to control Type-I error • Missing data are common; prospective plan to deal with missing data is not common • Exclusion of “outliers” is common; prospective plan to define and exclude “outliers” is not common

9. Example 1

10. Example 1

11. Example 2 No statistical comparison between treatment groups:

12. Do Editors of Journals Play a Role? • Editors/referees often do not demand an accounting of the details of the study – the study plan or the results • Limitations section is disingenuous: writers typically ignore the REAL limitations; write about limitations of the model to predict human disease (dah!), or point out a limitation they are already addressing in a new study, “setting the table” for the subsequent study

13. What Should be Done?

14. Fix these problems!Questions?