Research Ethics (and Avoiding Research Misconduct)

1. Research Ethics (and Avoiding Research Misconduct) National Naval Medical Center (NNMC) Graduate Education and Research Training Modules Linda D. Youngman MS, PhD • Office of Research Integrity (ORI) July, 2010

2. Disclosure Statement This CE/CME activity does nothave commercial support and has noconflicts of interest.

3. Learning Objectives • To learn basics of research ethics including: - interpersonal responsibilities - institutional responsibilities - professional responsibilities - ethical treatment of animals in research - human participation in research • To understand why integrity in research is absolutely essential. • To review case studies of research misconduct. • To learn the process for identifying and dealing with clinical research misconduct.

4. Research ethics – some basic principles • Research ethics involves the application of fundamental ethical principles to research. • Scientific research is built on a foundation of trust. • Protecting animals & humans in research studies is essential. • Key ethical principles: • Honesty Integrity Responsible Publication Carefulness Openness Respect for Colleagues • Confidentiality Privacy Objectivity • Responsible Mentoring Non-Discrimination Legality • Competence Social Responsibility • Animal Care Human Subject’s Protection Do unto others as you would have them do unto you.

5. Research ethics – a few definitions • Ethics – norms for conduct • Compliance – researchers and institutions follow the rules • Fabrication – making up data or results and recording or reporting them as factual • Falsification – manipulating research material or changing/omitting data such that the research is not accurately represented • Plagiarism – appropriating another person’s ideas, results or words without giving appropriate credit • Peer review – review by an objective, similarly-trained outsider (without vested interest in the outcome) • Research Integrity Officer (RIO) – institutional official responsible for assessing allegations of research misconduct • Respondent – person who is alleged to have committed research misconduct • Sequestration – removing & copying research data to protect evidence (avoid data alterations during inquiry/investigation) • Whistleblower – person who reports allegations of research misconduct in good faith

6. Research ethics – interpersonal responsibilities • Mentor / trainee relationships • - mutual respect and professional interest • - clear and direct communication, honesty • - neither trainee or mentor “owns” data (institution “owns” but mentor and trainee normally have copies) • Data retention and reporting • - keep good notebooks/documentation of research • - repeat experiments, use statistics properly • - accurately record & report data, don’t hide negative results • Collaboration and competition • - keep information open w/in group and easily shared • - credit determined by clear criteria that apply to all • - collegiality, ethical behavior is essential adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

7. Research ethics – interpersonal responsibilities • From: Director Drake Subject: General Research Rules • Welcome to our laboratory. Our research is funded by grants from NIH & other • agencies. Our projects have specific aims & a detailed research plan. Students • are not free to pursue research of their own, unless these fit the aims of the • funded project. Please be certain that the following are acceptable. • I agree to provide, as long as grant funds are available: • 1. Your tuition and a stipend to live on • 2. Laboratory facilities, including computers, equipment and desk • 3. Superior research training, a thesis idea and guidance • 4. A long-term commitment to your career goals • You agree that, since our priority is continued funding, I may: • 1. Set your daily work schedule and determine your research • 2. Present your research whenever & wherever I deem appropriate • 3. Decide what and when to publish • 4. Decide the authorship and order of names on all publications • 5. Determine your readiness for PhD qualifying and final exams • 6. Have ownership of your data -- before & after you leave the group adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

8. Research ethics – record-keeping • Data collection and recording • - hard copy data kept in a numbered, bound notebook • - electronic data should be validated (so it cannot be changed) and dated • Data protection & confidentiality of data • - data protected and accessible only to those who should have access • Data storage • - lab notebooks stored safely, computer files backed up regularly • - samples should be appropriately saved so they do not degrade over time • Period of retention of data & data sharing • - NIH requires data be kept for 3 yrs (btwn 3-7 yrs for some institutions) • - NIH’s view - data should be made as widely and freely available as possible while safeguarding the privacy of participants • Ownership of data • - data control stays w institutions (usually researchers have copies also) Good record-keeping protects you – take good records, keep them safe! adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

9. Research ethics – institutional responsibilities • Reviewing allegations of research misconduct • - encourage researchers to report suspected misconduct & in good faith • Protecting whistleblowers and respondents • - institutions have a duty to protect whistleblowers & respondents • Conducting inquiries and investigations • - research materials, data should be sequestered immediately • - inquiries are fact-finding missions to determine if there is evidence of misconduct (60 days) • - investigations determine if misconduct occurred (120 days) • Avoiding conflicts of interest • - researcher’s actions should not be affected by personal gain • - institutions must ensure that research misconduct committee members are objective • 5. Official institutional committees dedicated to research ethics • - IACUC or Institutional Animal Care and Use Committee (protect animals) • - IRB or Institutional Review Board (protect humans) adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

10. Research ethics – professional responsibilities • Proposing research and soliciting external funding • - PI’s should not claim results they don’t have yet to make grant applications more exciting • - for research ethics, grant applications are similar to publications • - contracts should resolve intellectual property issues before initiation • Presentations and publications • - present results clearly and accurately (& avoid dropping data) • - include sufficiently detailed descriptions so experiments can be replicated • Data retention and access • - data and notebooks must be sufficiently detailed to verify results • - data must be maintained for at least 3 yrs after project completion • Peer review • - publication in a peer-review journal indicates acceptance in the research community but does not verify that the researcher got it right adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

11. International Committee of Medical Journal Editors (ICMJE): All persons designated as authors should qualify for authorship and all those who qualify should be listed.... (pg 135, Intro Resp Cond of Research) Authorship credit should be based only on 1) substantial contributions to conception or design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically; and 3) final approval of the version to be published. Authors should describe what each contributed... Research ethics – authorship • Council of Science Editors – authorship checklist (pg 140, Intro Resp Cond of Research): Authorship contributions Resources Literature search Concept Material Writing Design Data collection / processing Critical review Supervision Analysis / interpretation Other Be a responsible author - with authorship based on substantial contributions only. adapted from: Introduction to the Responsible Conduct of Research – written by Nicholas Steneck, financially supported by ORI

12. Research ethics – ethical treatment of animals in research • Animal pain and distress must be minimized • - eliminate pain/distress if possible, use procedures to minimize pain/distress • Alternative methodologies are considered • - replacement – use in vitro or computer models, less sentient animals, microorganisms or plants when possible • - reduction – use animals or data gained from animals in more than one way • - refinement – construct experiments so that animal pain/distress is minimized • Ethical obligations for researchers using animals • - all animals, regardless of species, are treated humanely • - PI must ensure there are sufficient funds for proper animal care and use • - instruction should ensure that a minimum number of animals are used • Role of the IACUC • - the IACUC must review all animal research projects prior to their receiving funding and prior to any work with animals adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

13. Research ethics – protection of humans in research • Informed consent / Privacy / Confidentiality • - informed consent requires subjects know what will happen to their data • - privacy refers to a subject’s right to control what others know about them • - confidentiality refers to the care researchers must take with data collected from subjects • - subjects should not be coerced to participate in research • 2. Vulnerable populations and justice • - women, racial minorities, pregnant women, cognitively impaired individuals, prisoners, students and employees – special protection r’qd • Institutional Review Board (IRB - human subjects) • - IRB’s provide oversight for the conduct of research in human subjects • Must include for IRB review • - research background, human subjects, intervention, interaction, minimal risk, private information, “generalized knowledge” adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

14. Why ethics in research is so important • CASE STUDY #1: Dr. Leyos’s group has been studying a gene that may be • involved in helping tumors to metastasize. Three pilots have shown positive • results, but none is publishable. • First pilot - investigators were not blinded to the origins of the samples. • Second pilot - adequate controls were not included. • Third pilot - the freezer thawed resulting in some sample degradation. • Despite these problems, results from all three pilots were consistent. Based on • these results, a definitive experiment was done requiring 6 months with many • animals and dollars spent. On completion, Dr. Leyos learned that labels fell off • two samples, one for a control group and the other for the experimental group. • If the two samples are omitted, the results are not statistically significant (P<0.05). • If the samples are assigned to control & experimental groups one way, the analysis is not statistically significant & not consistent with the pilots. • If the unlabeled samples are assigned the opposite way, the analysis is statistically significant and consistent with the pilots. • Dr. Leyos must decide: • Repeat the experiment (taking 6 mos, 300 animals, and at a cost of $40,000) • Attempt to publish the results omitting the questionable samples • Assign the 2 samples to their likely groups and publish the results adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

15. Why ethics in research is so important • CASE STUDY #1: • - Repeat the experiment (taking 6 months, 300 animals and at a cost of $40,000). • Repeating the experiment is an ethically permitted and ethically encouraged approach. One of the key responsibilities of scientists is to achieve accurate and reproducible results. Whatever the reasons, unless the experiment is repeated, Dr. Leyos cannot attest that the results are accurate or reproducible. • However, the grant that funded this project did not budget for repeat of the experiment. So while Dr. Leyos wants to repeat the experiment, he must now decide among a number of courses of action. • Choose one of the following: • Take $40,000 (that was supposed to support a post-doc) and use the money to cover the additional experiment • Apply for other funding to repeat the experiment. In the proposal, Dr. Leyos explains that the results are currently just shy of statistical significance, but does not give details about the earlier problems • Explain the problems that have occurred to the funding agency and request additional funds to support the testing required for statistical significance adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

16. Why ethics in research is so important • CASE STUDY #1: • - Explain the problems that have occurred to the funding agency and request additional funds to support the testing required for statistical significance. • This choice is ethically permitted. Additional funds may be requested from the funding agency with explicit description of how funds would be used and why addtnl funds are needed. • Dr. Leyos does not necessarily have a duty to detail the catastrophes that have caused the results to be shy of statistical significance and, thus, resulted in the new request, but it is ethically permitted, perhaps even ethically encouraged, to offer the full explanation. Other researchers build on published data. If published results are false, it enormously slows down scientific progress. Scientific advancement is based on trust – be a trustworthy researcher. adapted from: http://ori.hhs.gov/education/products/montana - Online research ethics course, developed by Univ of Montana, ORI funded

17. Percentage of scientists who say they engaged in listed behavior within the previous three years (n=3,247) • Top 10 behaviors All Mid- Early-career • 1. Falsifying or ‘cooking’ data 0.3 0.2 0.5 • 2. Ignoring major aspects of human-subject rqmts 0.3 0.3 0.4 • 3. Not disclosing collab w firms whose products 0.3 0.4 0.3 • are based on one‘s own research • 4. Relationships with students, subjects or clients 1.4 1.3 1.4 • that may be questionable • 5. Using another’s ideas w/o getting permission 1.4 1.7 1.0 • or giving due credit • 6. Unauthorized use of confidential information 1.7 2.4 0.8*** • 7. Not presenting data that contradicts one’s view 6.0 6.5 5.3 • 8. Circumventing aspects of human-subject rqmts 7.6 9.0 6.0** • 9. Overlooking use of flawed data, questionable 12.5 12.2 12.8 • interpretation • 10. Changing design, methodology or study results 15.5 20.6 9.5*** • in response to funding source pressure • Engaging in 1 (or >1) of 10 behaviors (past 3 yrs) 33.0 38.0 28.0*** Note: significance of X2 tests of differences between mid- and early-career scientists are noted by ** (p<0.01) and *** (p<0.001). From: Martinson, Anderson and de Vries, Scientists Behaving Badly, (2005) Nature 435:737-738.

18. Percentage of scientists who say they engaged in listed behavior within the previous three years (n=3,247) • Other behaviors All Mid- Early-career • 11. Publishing same results in 2 or more • journals or publications 4.7 5.9 3.4** • 12. Inappropriately assigning authorship • credit 10.0 12.3 7.4 *** • 13. Withholding details of methodology or • results in papers or proposals 10.8 12.4 8.9 ** • 14. Using inadequate or inappropriate • research designs 13.5 14.6 12.2 • 15. Dropping observations or data points from • analyses based on a gut feeling that they • were inaccurate 15.3 14.3 16.5 • 16. Inadequate record-keeping related to • research projects 27.5 27.7 27.3 To protect the integrity of science, we must look beyond falsification, fabrication and plagiarism, to a wider range of questionable research practices. Note: significance of X2 tests of differences between mid- and early-career scientists are noted by ** (p<0.01) and *** (p<0.001). From: Martinson, Anderson and de Vries, Scientists Behaving Badly, (2005) Nature 435:737-738.

19. Why Images? ORI Cases With Questioned Images 44.1% 30.3% 40.4% Number of Image Cases Opened 14.3% 13.5% [55%] 5.7% 2.5% 4.1% ‘93-94 ‘95-96 ‘89-90 ‘99-00 ‘01-02 ‘03-’04 ‘97-98 ’05-06 [ to 4/07] 2 Year Reporting Period (’89-90 to ‘05-06) ORI has increasing numbers of research misconduct cases which involve falsified, altered or inappropriately manipulated images % of total cases 4-07, N = 100

20. Example – the same photograph used to represent two different treatments

21. Example – the same image used in two different figures DIO’s forensic Photoshop overlay analysis shows the two images are too similar to be different Original published figure – B6 Aire +/- and B6 Aire +/+ for Thymus The respondent later admitted that he used the same image for both the B6 Aire +/- and +/+ images above (selectively modifying them to make a “more beautiful picture”).

22. Example – the same image used in two different figures Fig 1 DIO’s forensic Photoshop overlay analysis shows the two images are too similar to be different (when aspect ratios are matched). XXXX Fig 2 (stretched horizontally) XXXXXX XXXXXX

23. Example – published image also had unwanted bands removed XX ←Original autoradiogram that was used for both previous figures Published figure was: stretched to make it look different from another figure (blue boxed area) had unwanted bands removed (red boxed area) XXXXXXX XXXXXXX

24. Example – published figure created from different autoradiograms (w bands removed) Published figure below was: created from different autoradiograms stretched to make it look different had unwanted bands removed (green boxed area) XX XXXXX XXXXX XXXXX XXXXX XXXXX XXXXX

25. Guidelines for images (but, if in doubt, call ORI) • Know limitations for capturing, storing & manipulating images • Images for comparison should be taken w identical conditions • Intensity measurements of images should be performed on raw data (& calibrated to a known standard) • Manipulation of images should be done w copy of raw image • Simple adjustments to the entire image are usually acceptable • Cropping an image is usually acceptable (if it doesn’t alter interpretation of image) • Manipulations specific to one area of an image, and not performed on other areas, are questionable • Software filters to improve image quality is usually not advised • Cloning objects into an image, or from other areas, is very questionable (e.g., “creating” a new gel band is unethical) • Avoid the use of lossy compression (& do not open/close JPEG multiple times) • Resolution/magnification should be set before capturing images • Be careful when changing the size (in pixels) of an image From: ORI working group and Cromey, D, Digital Imaging Ethics (2004) Southwest Environmental Health Sciences Center, Arizona

26. Guidelines for images • If you’re unsure about changing an image, check image guidelines – it’s better to be safe than sorry • Images for comparison should be taken under identical conditions • Cropping an image is generally OK • Altering one part of an image, but leaving other parts unchanged, is questionable • Cloning objects into an image is not acceptable Beautiful images are not as good as true images – if you’re unsure about alterations, it’s better to check or leave it unchanged. From: ORI working group and Cromey, D, Digital Imaging Ethics (2004) Southwest Environmental Health Sciences Center, Arizona

27. Misconduct in human trials – often more visible

28. Types of scientific misconduct in clinical trials from self-reports of clinical trial experts • Over-interpretation of “significant” findings in small trials • Selective reporting based on p values • Selective reporting of outcomes in the abstract • Subgroup analyses done without interaction tests • Negative or detrimental studies not published • Putting undue stress on results from subgroup analyses • Inappropriate subgroup analyses • Selective reporting of (1) subgroups, (2) outcomes, (3) time points • Selective reporting of positive results or omission of adverse events data • Failure to report results or long delays in reporting • Post hoc analysis not admitted • Giving incomplete information about analyses with non-significant results • Analysis conducted by the sponsor of the trial From: David Coultas, Ethical Considerations in the Interpretation and Communication of Clinical Trial Results Proc Am Thorac Soc Vol 4. pp 194–199, 2007

29. Types of data falsified/fabricated in clinical research: ORI 1993-2001 Interviews Entry criteria Screening logs Approval forms Follow-up exams / data Consent forms Test scores Laboratory results Patient data Number of subjects Dates of procedures Protocol Study results

30. Some examples of falsified clinical data: ORI 2007-2009 • Substituting one subject’s record for another • Altering dates and results for required eligibility tests • Backdating patient records to fit study protocol • Falsifying patient’s test results • Selective suppression of results or reporting data from a sub-set of patients only • Falsely reporting a larger “n” or number of subjects or falsifying outcome of statistical tests • Selective “omitting” of data • Using the bloods/tissues from one subject to represent two or more additional subjects • Falsifying interview results (interviews were not conducted) • Falsifying breath analyzer results (recording “0.00” when test was not conducted) • Representing animal data as if it were from humans • Falsifying images in presentations and published reports

31. ORI example - falsified clinical trial record • Aim was to increase enrollment into the clinical trial • Pelvic exam (which was rqd) had not been done • Laboratory tests (rqd post-operatively) were not done _____ Pelvic exam had not been done Laboratory tests were not done – forms were initialed and corrected later

32. ORI example of falsification – altering date of test to fit protocol’s time eligibility window XXXX XXXXX XXXX altered patient record – making it appear scans were done on 6-3-05 real patient record – showing scans were done on 3-26-05 true false

33. Misconduct in human trials –requirements for human subjects’ protection sometimes poorly understood

34. IRB, OHRP issues that are not research misconduct(in the absence of falsification/fabrication): • Failure to report adverse event to IRB/sponsor • Protocol deviations (entering ineligible participant or using off-protocol drug) • Forging physician’s signature on orders • Failing to obtain or properly document informed consent • Breaching human subject confidentiality • Withdrawing blood/tissue consent IRB = Institutional Review Board OHRP = Office for Human Research Protections

35. ORI OHRP Very often, both problems (ORI research misconduct and OHRP violations) are found in the same trial.

36. Example of falsification – handwritten test date (to fit protocol’s time eligibility window) XXXXXXX XXXXX XXXXX patient test date is handwritten as 9-28-04 actual patient test date was 10-29-04 XXXXX false true This is an ORI issue

37. Falsified withdrawalof consent for blood/tissue to be taken – OHRP issue, not ORI issue signatures copied from other consents XXXXXXXXXXX XXXXXXXXXX date stamped patient initially gave consent for blood/tissue to be taken XXXXXXXXXX

38. Sloppiness in informed consent – date is overwritten rather than striking through and initialed with new date – OHRP issue, not ORI issue XXXXXXXXXX XXXXXXXXXXX July 7 overwritten to July 6 XXXXXXXXXX

39. ORI’s totals for cases and accessions – 1992-2007 • About 250 accessions and • 30-40 cases each year

40. Investigations involving clinical research: ORI 1993-2007 • Total of all ORI cases (’93-’07) 475 • - percent clinical cases 85 18% • All cases - % misconduct 190/475 40% • Clinical cases - % misconduct 61/85 72% • A much higher percentage of ORI’s clinical cases (72% vs 40%) result in misconduct findings

41. Circumstances that led to detection of clinical misconduct in ORI cases • Detection by co-workers or colleagues 14 31% • Detection by an external quality control group (data coordinating center, external audit) 12 27% • Discovery by direct supervisor or grad. advisor 9 20% • Detection by an institutional “quality control” group (separate from supervisor) 7 16% • Detection by others (patients, IRB member, etc) 3 7% Total 45

42. Position of respondents against whom misconduct was found in clinical research: ORI 1992-2008 Professor 4 Associate professor 5 (24%) Assistant professor 2 Post-doctoral fellow4 (4 categ) Research assistant/associate 10 Nurse 2 (22%) Clinic coordinator2 (3 categ) Technician 23 Student 7 (54%) Other 4 (3 categ) Total 63

43. Lessons from ORI clinical cases - 1 • Any person involved in clinical research may be responsible for research misconduct, regardless of rank or duties on the project • The majority of respondents in ORI clinical cases are in technical positions (not doctoral degree holders) • Many of these respondents worked with inadequate supervision or training • Respondents may have had excessive work loads or time pressure - or pressured to increase enrollment • Both research misconduct and human subjects’ protection violations may occur in the same study

44. Lessons from ORI clinical cases - 2 • There are many different motivations for F / F: • financial (enrollment or accrual bonuses) • professional advancement (papers, grants, • tenure, job promotion / change) • personal • misguided altruism • Falsification / fabrication is often accompanied by sloppiness or carelessness in recordkeeping • Misconduct commonly discovered by: • Routine data audit • Whistleblower • Substitute staff member

45. Suggestions for preventing misconduct in clinical trials –communication & staff • The PI, Lab Director, etc. should establish a climate where scientific integrity (and the reasons for it) are emphasized – staff comfortable to voice concerns • All staff should be thoroughly trained in integrity principles and in their portion of the study • The PI and other supervisors should maintain strong communication with staff and a “presence” in the study setting, verifying personally at least a sampling of the research records each year • In particular, maintain strong lines of communication with your CRCs (clinical research coordinators) • In continuing studies, if possible, train more than one staff member to do follow-up

46. Suggestions for preventing misconduct in clinical trials – accurate records • Staff should be questioned about data alterations in the research record (overwrites, erasures, whiteouts, changes in electronic records) • Alterations on data forms must be done by striking through the original entry (no whiteout or writing over original entry), then initial and date the new entry • Copies of all laboratory reports should be retained by originating facility and coordinating center (to be spot checked on routine and special audits) • Keep staff work loads reasonable (so staff have enough time to stay on top of records)

47. Suggestions for preventing misconduct in clinical trials – keep it simple & audit • Research forms (registration forms, informed consent, etc.) should be as simple as possible, with realistic goals for staff / patients to complete them • Protocol sponsors should avoid paying bonuses based on number of patients enrolled • For interview studies, if possible, contact patients again for quality control reasons, and follow up • Audit, audit, audit – at least yearly, conduct spot checks of patient/research records (Navy conducts yearly audits) Keys to success in clinical trials – good communication, accurate records, keep it simple and audit.

48. What to do if the worst happens? – suggestions for handling clinical misconduct • Freeze enrollment immediately, particularly if the misconduct could possibly endanger patients • Stay calm – it may not be as bad as it seems (and it may not involve misconduct) • Enlist the aid/advice of your institution’s RIO (research integrity officer) • Keep the IRB (Institutional Review Board) informed • Sequester records promptly • Conduct a thorough investigation (not just the aspects identified by the whistleblower) • If in doubt, call ORI for advice (and check ORI’s website for clinical research misconduct guidelines)

49. Take home messages – keys to success • Do unto others as you would have them do unto you. • Good record-keeping protects you – take good records, keep them safe! • Be a responsible author – with authorship based on substantial contributions only. • Scientific advancement is based on trust – be a trustworthy researcher. • Beautiful images are not as good as true images – if you’re unsure, it’s better to check. • Keys to success in clinical trials – good communication, accurate records, keep it simple and audit.

50. If in doubt, call ORI for advice US Department of Health and Human Services Office of Research Integrity (ORI) Suite 750 1101 Wootton Parkway Rockville, MD 20852 240-453-8800 (tel) 301-594-0039 (fax) AskORI@hhs.gov (general email questions)