Enhancing Access to Data in Psychology

1. Enhancing Access to Data in Psychology Linda Beebe Senior Director, PsycINFO

2. Data may be the next king. . . • Last decade endless arguments about the king—Content? Technology? Relationships? Connections? • Now big focus on data—data repositories, even data papers. • But access issues & metadata standards not resolved, particularly for psychology. ALA MidWinter January 2012

3. Some Reasons for Data Sharing • Contributes to improving science. • Documents use of grant funds. • Now expected with NIH and NSF requirements for data management plans. • More efficient use of resources. • May increase knowledge of authors’ work, perhaps even add to their citation rates. • AND—it can reduce data fraud. ALA MidWinter January 2012

4. Psychology & Data Management • Especially in the natural sciences, traditions of sharing and data repositories such as Crystallography, ChemStar, PANGAEA, Dryad. • Also the Protein Data Bank, now one entity merging banks from the USA, Europe, and Japan. • Genbank in Medicine. • Psychologists do not have this tradition. • The APA Ethics Code states “psychologists do not withhold data. . .from other competent professionals who seek to verify the substantive claims through reanalysis. . .” And it calls for those reanalyzing data to use it only for that purpose. ALA MidWinter January 2012

5. Data may be scattered on laptops, in file drawers, on the web—all in danger of being lost, even thrown away. ALA MidWinter January 2012

6. And there is fraud. . . • Fraudulent data in science not new—Charles Babbage in 1830: “several species of impositions that have been practiced in science. . .hoaxing, forging, trimming, and cooking.” • Modern rubric is “fabrication” and “falsification, but it remains harmful and wasteful. • And for psychology, some recent incidents are very disturbing. ALA MidWinter January 2012

7. DiederichStapel • Social psychologist in the Netherlands, highly prolific and very popular with students. • Wrote with others, but generally volunteered for data collection. • Fraud identified by young researcher. • Coordinated committees at the University of Amsterdam, Tilburg, and Groningen examined publications from 1994 to 2011. • Also reviewed dissertations of his students defended in Groningen and Tilburg. • Late October they jointly issued findings of massive fraud, several dozen publications at a minimum. ALA MidWinter January 2012

8. What kind of fraud? • Created data from results of experiments with people ultimately found not to exist. • When he did conduct research, he manipulated the results. • Committee found 3 types of fraud: • Complete fabrication of data • Augmentation of a database when it appeared it would not confirm his hypotheses • Modification of complete or incomplete data ALA MidWinter January 2012

9. “. . . people, and in particular young researchers entrusted to him, have been affected profoundly at the start of their careers. This conduct is deplorable, and has done great harm to science, and the field of social psychology in particular. To the best of our knowledge, misconduct of this kind by a full professor in his position is unprecedented.” Interim Report, Tilburg October 31, 2011 ALA MidWinter January 2012

10. Unprecedented? Dipak K. Das • Professor of Surgery, Director of Cardiovascular Research Center, University of Connecticut Health Center • Prolific researcher with >500 articles, including 117 on resveratrol • Found guilty 1/11/2012 of 145 counts of fabrication or falsification of data. • 60,000-page report • U.S. Office of Research Integrity examining. ALA MidWinter January 2012

11. Costs are enormous. . . • Human lives—young researchers tainted, perhaps deprived of PhDs • Science—lack of trust in scientific studies. • Potential physical or mental harm from following researcher’s recommendations. • Wasted $─across many areas: • Publishers—original publication, all the retractions • Universities—CT spent 3 years investigating • Loss of grants. (nearly $1 million in federal funds for the University of Connecticut) • Follow-on research based on erroneous data ALA MidWinter January 2012

12. Data Sharing Prevention? • Stapel investigators determined fraud probably detected earlier if data had been publicly available. • Consciously constructed data does not look the same as random or collected data. • Person unconsciously introduces regularity that can be visually identified—distribution patterns give fraudulent data away. • J. Brean in National Post: “. . .without release of raw data and methodology, this kind of research amounts to little more than “trust me” science in which intentional fraud and unintentional bias remain hidden.” (12/30/11) ALA MidWinter January 2012

13. APA Task Force on Data Sharing and Data Linking in March 2011 • Small but diverse group with APA’s Chief Editorial Advisor, representatives from “big science” and “small science,” the APA Publisher, and the ED of the APA Science Directorate. • Reviewed changes in the conduct of science and technologies that enable data linking and sharing. • Recommended that APA develop a plan to put APA journals in forefront of data sharing. • Urged consideration of teaching psychology students how to adapt to the culture change of shared data resources. ALA MidWinter January 2012

14. Second APA Task Force in September 2011 • Developed draft principles for data sharing in APA journals, which the Publications & Communications Board approved “in principle” in November. Essence: • Data sharing promotes science. • Participants’ privacy must be protected and data kept confidential. • Original author and secondary user share responsibility for that protection. • Secondary user must acknowledge original source and may not transfer data to others. ALA MidWinter January 2012

15. Result: New APA Journal in 2012 • Broad psychology journal, not niche. • Experimental in delivery—electronic only; multimedia, executables, etc. welcomed for inclusion in articles. • Requirement: Author must agree to share data. • Co-editors: Gary R. VandenBos (APA Publisher) & Harris Cooper (APA CEA) ALA MidWinter January 2012

16. New Journal Requires New Expectations • To enable data sharing, authors must: • Provide clear metadata. • Code consistently. • List coding instructions. • Detail data collection. • Explain how data were used. • Provide all raw data. • Organize data in a way can be used by others. • Methods sections written differently: • Meet CONSORT (CONsolidated Standards of Reporting Trials) guidelines for randomized controlled trials. • Meet JARS (Journal Article Reporting Standards) guidelines. ALA MidWinter January 2012

17. Biggest Concerns: Data Description • May describe only the data and parameters. • May neglect to describe population, recruitment processs, final samples, drop-out rates. • Therefore Methods section will be inadequate. Access • Metadata • Access methodology Credit for Original Author • How acknowledge the data author? Authorship? Contributor? Data Collector? Shared Responsibility for Data • Protection of privacy and confidentiality. • Accuracy. ALA MidWinter January 2012

18. Honest Researchers Worry about Sharing Their Data. . . • Potential nefarious uses─unscrupulous people could twist data or hector the people author is trying to help. • Well-intentioned but inept secondary analysis—they might get it wrong! • Loss of potential publications for self—I don’t have all my articles written from this data! • Most common fear—loss of academic credit for what may be years of data collection. Acknowledgement of original author will be essential. ALA MidWinter January 2012

19. Repository for APA Collected Data • Exact shape as yet undetermined. • VandenBos initial meeting with 8 psychology organizations this month. • Will not be on open web (as APA handles Supplemental Material for its journals). • Will be indexed with descriptive metadata. • Potential users will need to apply for use and probably sign some declaration of intent. ALA MidWinter January 2012

20. Note on Supplemental Materials • Two-year effort of NISO-NFAIS Working Group to develop Recommended Practices. • Business Policies & Practices cover selecting, editing, hosting, assuring discoverability, referencing, packaging, maintaining links, providing context, and preserving—not data. • To be released for public comment early February. • Technical Recommendations for implementing to follow very soon. • www.niso.org ALA MidWinter January 2012

21. Citing Datasets • Another major area of concern. • APA Style specifies how to cite unpublished raw data. • However, issue is facilitating access to at least a descriptive summary, also use by software tools. • JISC document “How to Cite Datasets and Link to Publications” excellent resource. http://www.dcc.ac.uk/node/9269 (Digital Curation Centre) • Can be sure APA “style mavens” and our metadata experts will work on this. ALA MidWinter January 2012

22. Thank you! Linda Beebe Senior Director, PsycINFO American Psychological Association lbeebe@apa.org www.apa.org 202-336-5636 ALA MidWinter January 2012