Teaching Ethics in Statistics Class

1. Teaching Ethics in Statistics Class John H. Walker Department of Statistics Cal Poly, San Luis Obispo jwalker@calpoly.edu

2. Outline • Ethical Guidelines for Statistical Practice (ASA, 1999) • Some thoughts on the JSM session • Teaching ethics at Cal Poly • A statistical pet peeve Teaching Ethics in Statistics Class

3. ASA Ethical Guidelines: Overview • Prepared by the Committee on Professional Ethics • Approved by the Board of Directors • How many have read them? • Eight sections: • Professionalism • Responsibilities to Funders, Clients, and Employers • Responsibilities in Publications and Testimony • Responsibilities to Research Subjects • Responsibilities to Research Team Colleagues • Responsibilities to Other Statistician or Practitioners • Responsibilities Regarding Allegations of Misconduct • Responsibilities of Employers Teaching Ethics in Statistics Class

4. Ethical Guidelines: Professionalism • Strive for practical relevance in statistical analyses • Guard against predisposition about results • Remain current in statistical methodology • Assure adequate statistical and subject-matter expertise • Use only methodologies suitable to the data • Do not join a research project unless you can expect valid results and your name is not used without consent • Understand the theory, data, and methods behind automated procedures • Recognize the implications of multiple frequentist tests • Respect and acknowledge the contributions of others • Disclose conflicts of interest and resolve them Teaching Ethics in Statistics Class

5. JSM Session: Teaching Ethics in Statistics Class George McCabe, Purdue Univ. “Ethics and the Introductory Statistics Course” Patricia Humphrey, Georgia Southern Univ. “Ethics, It’s for Everyone!” Paul Velleman, Cornell Univ. “Truth, Damn Truth, and Statistics” Journal of Statistics Education www.amstat.org/publications/jse/v16n2/velleman.html Teaching Ethics in Statistics Class

6. McCabe • “New Course” vs. “Old Course” • “Old course” ethics examples: • Decide the significance level before looking at the p-value • Make sure assumptions are satisfied. (Then what?) • Don’t say too much! • “Students are afraid to conclude anything” • “New course” ethics emphasizes: • Question formulation • Correct choice of method • Focusing on the data • Are we teaching the “new course” with “old” ethics? Teaching Ethics in Statistics Class

7. Humphrey • Data ethics • Much more than “have we avoided bias” • Institutional Review Boards • Confidentiality • Informed Consent • Case studies are great ways to teach ethics! • Continuous reinforcement throughout the class • Project data collection • Labs • Exams Teaching Ethics in Statistics Class

8. Velleman • Statistics is the honest search for truth about the world • Good statistics requires judgment • “The best analysis often arises from the Darwinian competition among alternative models.” Survival of the best fit In the end, there may not be a single “best” model • Public mistrust of statistics “The problem isn’t that another sample may give a different answer, but that another statistician working with the same sample may give a different answer.” Teaching Ethics in Statistics Class

9. Teaching Ethics in the Cal Poly Statistics Major • First quarter (“Concepts and Controversies” level course) • Debates on ethical issues (animal testing, informed consent) • Complete NIH online ethics training • http://researchethics.od.nih.gov • Late 1st/Early 2nd year (2 course Applied Stat sequence) • Choice of statistical method • Data collection • Type I & Type II errors, power • Multiple comparison methods • Assumptions and alternative methods • Limits of a statistician • Importance of subject matter knowledge • Practical significance Teaching Ethics in Statistics Class

10. Teaching Ethics in the Cal Poly Statistics Major • Third & fourth year (Electives, e.g. Survey Sampling) • Nonresponse in survyes • Class project • Institutional Review Board / Human Subjects Committee • Last quarter (Capstone: Communication and Consulting) • Team projects • Mock consulting sessions • “Unaided” choice of statistical method • Dealing with “pushy” clients • ASA Ethical Guidelines Teaching Ethics in Statistics Class

11. Teaching Ethics: Conclusions • Emphasize “judgment points” in data analysis • Discuss alternative choices and consequences • Study design • Observational study vs. designed experiment • Was the data collected ethically? • Assumption checking and reexpression • What is the possible effect of a violation? Of reexpression? • All assumptions are not created equal. Some are more important. • Multiple comparisons • How many tests did you run? Each p-value you look at is a test. • Outliers and influential observations • Identify and gather information. Are they real or errors? • How do they affect the results? • Disclose any changes to your data. Teaching Ethics in Statistics Class

12. Teaching Ethics: Conclusions • Focus on the data: an analytical outline • Look at the data. (Make a graph.) • Analyze the data. • Draw conclusions. • Look at the data again.Reevaluate conclusions. • Reporting results • State conclusions with authority (within reason). • Don’t confuse causation and association. • Report statistical significance (p-value). • Report practical significance (effect size and direction). • When possible report intervals, not just point estimates. • Report any unresolved problems and possible consequences. Teaching Ethics in Statistics Class

13. My (Current) Statistical Pet Peeve • Remember Ethical Guideline #8: Recognize the implications of multiple frequentist tests • Problem: Uneven application! • Why do we usually talk about multiple tests only when we teach ANOVA? • Classic example: One-way ANOVA with 4 factor levels 6 pairwise comparisons Standard multiple comparison methods control overall error rate Teaching Ethics in Statistics Class

14. What about… • Multiple regression • Multifactor ANOVA • Multiple response variables • Multiple regression or multifactor ANOVA with several response variables in the study • Canned multiple comparison methods do not control the overall Type I error rate in these situations. • Do we tell our students enough about this problem? Teaching Ethics in Statistics Class

15. Examples • A multiple regression with 10 predictor variables Each predictor tested at a= .05 10 tests. No control on overall Type I error rate • A three-factor experiment, each factor with 4 levels Each term tested at a= .05 with multiple comparisons at a= .05 3 main effects, 3 two-way interactions, 1 three-way interaction Canned MC methods will adjust for comparisons within each factor, but not across the different terms in the model. 7 tests. No control on overall Type I error rate • The above design with 5 univariate responses 35 tests. Yikes! Teaching Ethics in Statistics Class

16. Solutions • Bonferroni adjustment • Controls overall Type I error rate, but very conservative • Simple enough to use everyday—even in intro classes • Higher level classes could use more powerful step down versions • What about power? • Who says you have to have a 5% overall Type I error rate? • Before analysis, just choose a higher overall Type I error rate. • A pseudo-Bonferroni adjustment (working backwards) • Don’t like weird fractional individual significance levels? • Use a small, rounded comparison-wise rate. • Back compute the upper-bound on the overall Type I error rate. Teaching Ethics in Statistics Class

17. What To Tell Students • Be aware of the problem: Ignorance is not bliss! • Discuss the consequences of different approaches • Exploratory vs. confirmatory analyses • Balancing power and overall Type I error rate • What to do may be a judgment call • If you adjust, understand the power implications • If not, count the tests, then compute and report the Type I error bound • Stand up to clients who don’t want to adjust • Guiding principle: The honest search for truth about the world • Then, make sure we practice what we teach! Teaching Ethics in Statistics Class

18. Thank you! Teaching Ethics in Statistics Class