Engaging in Statistical Practice in Academia is Honorable

1. Engaging in Statistical Practice in Academia is Honorable Michael J. Schell Moffitt Cancer Center & Research Institute

2. The Methodologist and the Practitioner The roles of a statistical methodologist and a statistical practitioner, while overlapping somewhat, are distinct. Academia still needs to learn how to properly define the proper expectations and rewards for the statistical practitioner

3. FSU Builds a New Academic Building Circa 1978 Two of following four departments should move to the new building: Mathematics, Meteorology, Oceanography, Statistics. What split would be best? Apparently some departments are too academically too close for comfort.

4. The Farmer and the Cowman Should be FriendsOklahoma – Rodgers and Hammerstein Pro-farmer I'd like to say a word for the farmer, He come out west and made a lot of changes Pro-cowman He come out west and built a lot of fences, And built 'em right acrost our cattle ranges.

5. The Farmer and the Cowman Should be Friends Pro-cowman The farmer should be sociable with the cowboy if he rides by and asks for food and water.Don't treat him like a louse make him welcome in your house. Pro-farmerBut be sure that you lock up your wife and daughters!

6. Societal Costs to Poor Medical Training While systems designed to cull the best from the rest might be desirable in situations like baseball’s World Series, this ought not to be our design in the training of medical students. We want each medical school graduate to have mastered all the material considered important. At present, those doctors in need of remedial assistance are often not identified until some kind of ex-post screening process is undertaken, such as in-service examinations, specialty board certifications, or malpractice lawsuits. Innovator’s Prescription, p. 352-3

7. Medical Schools in La-La Land What roles are today’s leading medical schools playing in training the caregivers we’ll need? In fact, they’re training more and more of the doctors we won’t need, and leaving others to train the professionals we will need. The medical education system in the United States has no means of coordinated planning for training doctors for societal needs. Innovator’s Prescription, p. 354

8. Improper Reward Structure in Medical Education The only resource allocation mechanism is that medical students, like the rest of us, choose careers that are intellectually and emotionally engaging, with the most attractive incomes and lifestyles. … This has led to a rapid expansion of subspecialists and a dearth of primary care physicians. … As for the shortage of primary care physicians, America is turning to immigrants from foreign nursing schools Innovator’s Prescription, p. 354-5, 357

9. We anticipate that, given the need for more primary care doctors and nurses that our medical and nursing schools are not meeting, major integrated health-care providers will begin training their own caregivers. Innovator’s Prescription, p. 360 End Result: Disruption of Medical Schools

10. How Many Discoveries Have Been Lost by Ignoring Modern Statistical Methods?Rand R. Wilcox, American Psychologist, 1998 Arbitrarily small departures from normality result in low power; even when distributions are normal, heteroscedasticity can seriously lower the power of standard ANOVA and regression methods. … most quantitative articles tend to be too technical for applied researchers. If the goal is to avoid low power, the worst method is the ANOVA F test.

11. British Medical Journal articles by Doug Altman The scandal of poor medical research, 1994 Why are errors so common? Put simply, much poor research arise because researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them. Statistics and ethics in medical research. The misuse of statistics is unethical, 1980

12. Ethical Guidelines Discussion, (JH Ellenberg, TAS,1983) RD Remington: “The discipline of statistics, which emphasizes the development of new methodology and new theoretical structures, is perhaps comparable to its parent discipline, mathematics …” “The practice of statistics on the other hand, is a profession, involving at every stage judgment and informed choice.” “The guidelines themselves give only slight notice to a central feature of professional practice of any type – professional judgment.”

13. Ethical Guidelines Discussion (2) HV Roberts: “The Guidelines will help to remind statisticians … that statistical practice requires integrity as well as professional skill. But [the Guidelines] sound bland.” Temptations are pervasive, yet subtle: 1. to modify one’s best evaluation of the data by what the audience or client wants to hear. 3. to reject needed tools on the grounds that they will prove too difficult to explain. 4. to be lax in seeking out the most appropriate statistical tools. 7. to neglect checks and safeguards against data problems, model failure, and processing errors.

14. Marketing of Pharmaceuticals • Must have the produced the drug and shown its efficacy • Need to produce the drug in mass quantities • Marketing

15. Marketing of Statistical Ideas • Must have derived the statistic and demonstrated its efficacy • Need to have available software • Need to disseminate the idea

16. Key Principle In an environment where ideas are not marketed, first on the market wins

17. First-on-the-market winners Chi-square test T-test, 1908 ANOVA Kolmogorov-Smirnov test, 1937 Duncan’s test, 1950 Kaplan-Meier curves, 1958 Cox regression, 1972

18. Three examples where education and practice lags woefully behind • Two-sample location problem • 2 x 2 contingency table • Multiple testing adjustment

19. Multiple Testing Adjustment Traditional answer: Bonferroni Improved answer: Holm

20. Holm adjustment J or Scandinavian Statistics, 1979 3,432 total citations (3/16/09) 288 in 2007 386 in 2008 Personal thanks: Gary Koch and Alex Dmietrienko

21. Two-sample location problem Traditional answer: Wilcoxon rank sum test for small samples, t-test for large samples Better practice: Wilcoxon or normal scores test for large samples, t-test for small samples (such as n=3, but assumptions are critical!)

22. Hodges and Lehmann , 19614th Berkeley Symposium Hodges and Lehmann (1956) proved that the ARE of the Wilcoxon rank sum test is at least .864 Chernoff and Savage (1958) proved that the ARE of the normal scores test is at least 1 “The above results suggest that on the basis of power, at least for large samples, both the Wilcoxon and normal scores tests are preferable to the t-test for general use.”

23. First Simulation on Robustness of t-testCA Boneau, 1960 320 citations Assessed validity only, using three distributions, normal, uniform, exponential and three sample size pairs Conclusion: t-test is fine. Later discovery: exponential simulation was done wrong Highest citation count on any subsequent simulation study (39 articles thru 2000) = 96

24. Misconceptions Leading to Choosing the t Test Over the Wilcoxon-Mann Whitney Test for Shift in Location Parameter J of Modern Applied Statistical Methods, 2005 by SS Sawilowsky “The knowledge about the large sample asymptotic theory “had even penetrated to the level of a book review written in 1968!” “The Wilcoxon rank-sum test … show[s] only slight losses in both large and small sample efficiency relative to the t-test in the normal case, while in many non-normal cases, efficiency exceeds 100%”.

25. Who was the book review author? Duane Meeter! But who was listening?

26. On Student’s 1908 ArticleJASA March, 2008 Comment by Diaconis and Lehmann … even under slight deviations from normality, the t-test can be far from optimal. The poor performance of the t-test, particularly for distributions with heavy tails, can be seen in comparison with nonparametric tests, such as the Wilcoxon or normal scores tests. … for all distributions with finite variance, the asymptotic relative efficiency relative to t is ≥ .864 for Wilcoxon and ≥ 1 for normal scores.

28. Textbook Placement Basic Practice of Statistics, 4th Ed. 2006 David S. Moore (728 pages) Non-parametric tests don’t make the book; they appear in the virtual appendix. Statistics: A Biomedical Introduction, 1977 Hollander and Wolfe T-test in Chapter 5; Wilcoxon in Chapter 13 Biostatistics, 2nd Ed. van Belle, Fisher, et al., 2004 T-test in Chapter 5; Wilcoxon in Chapter 8

29. 2x2 contingency table Traditional answer: chi-square test unless expected cell count < 5, then Fisher exact test Better answer: It depends on the design (what marginals are fixed), For most practical situations: unconditional test

30. Problems with chi-square It cheats (is invalid) sometimes and we now can tell where by exact or Monte Carlo results. The rule of thumb reduces the violations but doesn’t eliminate them. D’Agostino, Chase, and Belanger recommend that one can use the chi-square test all the time! (TAS, 1988)

31. Enter Barnard’s unconditional test 1945,7 Barnard introduces test 1949 Barnard renounces own test 1979 Kempthorne: “The importance of the topic cannot be stressed too heavily .. 2x2 contigency tables are the most elemental structures leading to ideas of association. … It is remarkable that a consensus has not been reached”. 1985 Suissa and Shuster re-introduce test 1985? Mehta (at JSM conference) “I’m not sure I believe in it … but if people want it (chuckles), I might include it in StatXact.”

32. Recent Articles Campbell (Stat Med, 2007) recommends the Mantel-Haenszel chi-square when expected cell size are 1 or more Lydersen, Fagerland, and Laake (Stat Med, 2009) recommend the unconditional test unless both margins are fixed Proschan, and Nason (Bmcs, 2009) “One lesson is that when the design dictates the margins … one should condition on them.”

33. Ethical Considerations Ethical Guidelines for Statistical Practice, ’99 “The use of statistics in medical and biomedical research may affect whether individuals live or die” “… society depends on sound statistical practice … many unresolved issues that deserve frank discussion.” Educators have an ethical responsibility to properly train their “tool user” students in best practices “Tool user” statisticians have an ethical responsibility to seek best practice information

34. Insufficient Focus on Teaching Goals … students would rotate from one clerkship to the next in an identical sequence – rather than crisscrossing back and forth as they presently do, and where the quality of students’ educational experiences depends to a frustrating degree on “the luck of the draw”. Innovator’s Prescription, p. 348

35. Why Medical Schools Won’t Change Although there are clear needs for sustaining improvements to medical education, … we are pessimistic that our leading medical schools will be able to act decisively on either front. … The reason lies in the mechanisms of governance in these institutions – which are largely collegial and consensus-driven. Innovator’s Prescription, p. 358

36. Oklahoma Reprise Territory folks should stick together, Territory folks should all be pals. Cowboys dance with farmer's daughters, Farmers dance with the ranchers' gals.