Ethical Statistical Practice

1. Ethical Statistical Practice • Study Design • Data Collection (Human Subject Protection) • Data Management • Data Analysis

2. Study Design • It is unethical to waste money, resources, and time on shoddy research. • Appropriate selection of a control population; minimization of various sources of bias. • Collect correct outcome of interest, potential confounders, effect modifiers, other relevant data elements (e.g., for non-response estimation). • Minimize measurement error, missing data.

3. Sample Size Justification • It is unethical to carry out a study with many more subjects/animals than needed, or with not enough to possibly answer the question of interest.

4. Human Subjects Protection:Institutional Review Boards • Every University, hospital, or company where human or animal research is performed has an IRB. • The IRB is a committee that may include health care professionals, ethicists, statisticians, and lay members. • The IRB insures that research on humans or animals is done in an ethical manner. • Each study is reviewed before it starts, and is reviewed again if any change is made.

5. Monitoring of Clinical Trials • Most studies involving experimental treatment of human subjects should be monitored regularly. • Monitoring should be done by a separate data safety monitoring committee/board (DSMC/DSMB).

6. Data Safety Monitoring Boards (DSMB) • A DSMB is generally formed for clinical trials that are large, multi-center, or have a safety concern. • DSMB members (physicians, ethicists, statisticians, etc.) are not involved in the study execution, and are often outside the study institution(s). • The DSMB usually meets once or more a year and evaluates adverse events and study progress. • Statisticians often have important roles in DSMB discussions. • A key question the monitoring committee asks is “Should the trial be stopped to publicize benefit or to prevent harm?”

7. Ethical Issues in Study Termination • The tug between early dissemination of findings to help treat disease vs. continued study to gain more knowledge. • Strong evidence that the experimental treatment is more effective or less effective than the comparison treatment. • Special methods for dealing with multiple “peeks” at the data to retain proper Type I error.

8. Trials Stopped for Early Dissemination of Results • Tamoxifan Trial for prevention of breast cancer • Strong beneficial results in reducing risk of breast cancer • Possible increased risk of uterine cancer and blood clots in the lungs • Unclear target population for general use • Two smaller European studies found no reduction in breast cancer risk in women who took Tamoxifan. • In retrospect, doubt has been cast on wisdom of the rush to approve Tamoxifan for prevention.

9. Trials Stopped for Early Dissemination of Results • Women’s Health Initiative: hormone replacement therapy to reduce risk of heart disease. • Stopped 3 years early because of increased risk for breast cancer, heart disease, and stroke. • Reduced risk of fracture, colorectal cancer. • Many previous studies without randomization had found HRT protected against cardiovascular disease. • Continued analysis suggests lower risks and greater protective effects when women begin immediate after menopause.

10. Responsible Data Management • Security • Confidentiality • Accuracy • Documentation

11. Security • Password protection • Regular back-ups performed • Back-up copy off-site stored regularly • Maintain virus protection

12. Confidentiality • Employee responsibility • Name accessibility • Password access to data • Locked filing cabinets for paper records.

13. Health Insurance Portability and Accountability Act (HIPAA) • Use of protected health information (PHI) for research generally requires subject approval • names, birthdates, addresses, zip codes, phone #s, social security #s or other identifying #s, photos, biometric ID data. • Can share within research group “umbrella” • Use of “de-identified data” not included in the 18 types of PHI less restrictive. http://www.sph.umich.edu/faculty_research/hipaa/why_hipaa.html http://privacyruleandresearch.nih.gov/pdf/HIPAA_Booklet_4-14-2003.pdf

14. Data quality • Data editing of paper records • Double data entry • Computer checking of data

15. Computer checking of data • Range checks • Date checks • Consistency of data across questions • Document error correction

16. Documentation of the Database • Maintain a codebook for the database • List of all variables • All valid codes • Range of valid values • Consistent coding for answers such as Y/N • Use numbers, not text, where possible. • Record changes in the questions or answers, and dateimplemented

17. Responsible Statistical Analyses • Document all steps that produce the analysis • Save command files or point-and-click sessions • Archive data and analysis files.

18. Lies, Damn Lies, and Statistics • It’s often possible to bend a statistical analysis to our own point of view. We often do so unconsciously. • Journals don’t publish negative studies, so we have to report what’s significant. • People whose jobs depend on publications try to “find something interesting” in the data.  Journals are full of Type I errors.

19. Practices to Avoid • Fabricating data • Changing data • Deleting observations that don’t agree with your conclusion • Performing multiple statistical tests, and only reporting the significant ones

20. Selective Inclusion of Data • Reasons for deleting data: • Outliers; Influential points • Cases with missing data • Drop-outs; Non-compliers • Correct or delete verifiable data errors. • If outliers remain, present the analysis both with and without the offending points. • If missing data, consider sources of bias.

21. Multiplicity of Statistical Testing • Multiple endpoints • Multiple comparisons • Multiple looks over time • Subgroup analyses • Cornfield (1976): “Just as the Sphinx winks if you look at it too long, so, if you perform enough significance tests you are sure to find significance, even when none exists.”

22. Data Dredging as an Analysis Technique From Meinert, Clinical Trials: “The practice of data dredging is common . . . In fact, it is the hallmark of most epidemiological research concerned with identifying etiological factors of diseases.” • To guard against the dangers of Type I errors: • Specify major hypotheses prior to analysis • Don’t selectively report significant results • Consider multiple comparison corrections • Require confirmation with new studies.