The Social Cost of Scientific Misconduct and Questionable Research Practices

1. The Social Cost of Scientific Misconduct and Questionable Research Practices William Gardner, PhD Departments of Pediatrics, Psychology, & Psychiatry The Ohio State University Center for Innovation in Pediatric Practice, Columbus Children’s Research Institute Supported by NIH/ORI: 1R01 NS49591.

2. Social Costs of Research Misconduct (RM) • How much should we care about RM? • “A lot, because integrity is essential to science.” • But why should non-scientists care about this? • Consequentialist answer: “A lot, because research misconduct may have a large social cost.” • My talk is conceptual, not empirical. I’ll define this social cost and suggest how to study it.

3. Direct, Visible Economic Costs of Research Misconduct • Steneck: What does the NIH lose from RM? • ‘Direct’ Cost Loss = Grant Award $s. • ‘Visible’ Cost  Use only known RM cases. • Loss / Year = (N RM Cases/ Year) xCost / Case. • Cost / Case = $427K (Avg 2005 NIH award). • N RM Cases = 10 cases / Year. • Therefore, Loss / Year = $4.3M.

4. Direct, Invisible, Economic Costs of Research Misconduct • ‘Invisible’ Cost  Impute rate of RM cases based on hypothesized incidence of RM. • N of NIH Grants(2005) = 47,345.

5. Indirect Economic Costs of Research Misconduct • ‘Indirect’ Cost Loss = monetary value of benefits that society would have received from research, but did not. • Challenge 1: Measure monetary value of science. • Challenge 2: Identify the benefits that society did not receive because of RM.

6. Two Thought Experiments • Did research misconduct delay the start of smoking prevention efforts? • Does research misconduct reduce the efficiency of pharmaceutical research?

7. Thought Experiment: Scientific Misconduct & Smoking Massive behavior change, catalyzed by dissemination of scientific research.

8. Reduction of CVD Mortality Attributable to Smoking Reduction Based on Cutler (2001). 4.5 years improvement in life expectancy over 50 years due to reduction in CVD mortality. Of that, 1.5 years reflects behavioral change. Of that, assume .75 years is smoking reduction.

9. Additional Life-Years From CVD Mortality Reduction Attributable to Smoking Reduction Accumulate the per capita gain in chance of surviving a year across population.

10. Annual Value (in $B) of CVD Mortality Reduction Attributable to Smoking Reduction Valuing a year of life at $100,000 (Murphy & Topel, 2001).

11. What if the Surgeon General Spoke Out in 1959? Scientific debate about smoking was rife with misconduct. • Counterfactual curve: • Suppose that RM & QRP had not clouded the debate. • Suppose that prevention efforts had started 5 years earlier?

12. Additional Benefit From Reduced CVD Mortality Attributable to Earlier Smoking Reduction Shaded area is the value of the reduction of CVD mortality, had we started 5 years earlier ( > $100B). Moving prevention efforts forward 1 year saves $27B. Under these counterfactuals, these are the social losses attributable to RM.

13. Drug Development Process Reduced Morbidity & Mortality Investment Thought Experiment: Misconduct, QRP, & New Drugs RM & QRP results in the direct loss of investment. However, RM in the drug development may also reduce N of new drugs / $ invested. Fewer new effective drugs means less reduction in mortality. Indirect Cost = Value of life-years not saved.

14. The Drug Development Process • Drug development is a multiple stage process, from pre-clinical investigations through multiple phases of CTs. • Later stages of development involving animals and humans are far more expensive than earlier stages.

15. Misconduct & The Drug Development Process • Misconduct increases the False Positive Rate in the drug development process. • Small increases in the False Positive Rate in drug testing will substantially increase the proportion of useless compounds tested in expensive later stages. • This wastes research dollars, a direct cost of RM. • However, it also increases the investment cost of developing a useful drug  fewer drugs are developed.

16. The Value of the Missing Drugs • The indirect cost of RM in drug development are those missing drugs. • Econometric models suggest that better drugs for CVD have a larger impact than smoking reduction. • Hence the annual value of life-years saved due to improved CVD medication alone is worth tens of $B. • Small inefficiencies in the drug development process will result in losses that are likely to be scaled in $B.

17. Summary: Social Cost of Fraud in Pharmaceutical Research • RM and QRP make it harder to eliminate bad ideas early, and may induce other researchers to pursue false leads, reducing research efficiency. • Therefore, RM and QRP increase the cost of discovery, reduce the rate of discovery, and reduce the rate of growth of life expectancy. • RM & QRP are like securities fraud: Society loses from both the loss of the investment and reduced economic growth from the misallocation of capital.

18. So we have proved what, exactly? Buck Mulligan, in the Martello Tower, speaking of Stephan Dedalus: “It’s quite simple. He proves by algebra that Hamlet’s grandson is Shakespeare’s grandfather and that he himself is the ghost of his own father.” James Joyce, Ulysses I am waving my hands. The point of waving my hands is to point out a new question, and suggest how we might try to answer it.

19. Limitations of Thought Experiments • Our examples do not include data on misconduct. • Our arguments rely on economic models that are controversial. • We consider only medical examples.

20. Conclusion to Thought Experiments • We have clarified the meaning of an indirect social cost of research misconduct. • We have also clarified the stakes in RRI. • Steneck’s estimate of the direct costs of RM were expressed in $M. • I think the value of indirect social costs of RM should be expressed in $B (3 orders of magnitude).

21. Conclusion: The Social Costs of Scientific Misconduct • Because RM is relatively rare, the direct costs of RM are likely small. • The social value of science, however, is titanic. • Inefficiencies in technology transfer caused by RM should be projected across this large value. • Social costs of RM are greater than are imaginged, and vastly greater than the cost of research on RI.