Download
1 / 36
360 likes | 419 Views
AWHILES Study Day 13 May 2011. Julie Browne External Relations Manager, Wales Deanery, PGMDE University of Cardiff. “Published research: how do you know if it’s any good?”. …or.
E N D
AWHILES Study Day13 May 2011 Julie Browne External Relations Manager, Wales Deanery, PGMDE University of Cardiff
“The role of the editorial office in safeguarding the integrity and academic rigour of the scholarly archive, with particular reference to the medical education literature”
What I’d like to cover today • The role of journals in safeguarding quality • Medical education journals • Peer review – what is it? • Peer review – how is it done? • Peer review – why do it? • Publication ethics
“How do you know if a published paper is likely to be reliable?”
Quality in journals • Metrics • People • Content • Systems
Leg 1: Metrics • Citations (including self-citation) • Indexes • Rejection rates • Speed of peer review/acceptance to publication • Impact factor/immediacy
Leg 2: People • Editorial board, editor, editorial staff • Publisher • Learned society • Peer reviewers • Authors
Leg 3: Content • High quality papers (goes without saying – we’ll discuss this further in a minute!) • Letters and commentaries sections (post-hoc peer review) • Editorials and guidance on scope and mission • Retractions, corrections, errata • Participation in journalology research
Leg 4: Editorial Systems • Transparent processes, clearly announced – how we do things, what you can expect from us • Published policies: e.g. ethics, peer review, handling of complaints, handling of editors’ papers • Membership of COPE or other editors’ group • An ethics committee • A statement regarding editorial independence (e.g. “All content in Academic Medicine reflects the views of the authors and does not reflect the official policy of the AAMC unless clearly specified.” “The BMA grants editorial freedom to the editor of the BMJ. The views expressed in the journal are those of the authors and may not necessarily comply with BMA policy.”)
But, above all… A rigorous, effective and fair peer review system
“The purpose of peer review is to keep egg off authors’ faces”Drummond Rennie (JAMA)
The review process Completion of research Writing and preparation Submission Internal review External review Decision Revision Rejection Resubmission Re-review Acceptance Rejection Publication
Common reasons for rejection Did not address an important scientific issue Research question not clearly stated Has been done before Results are unsurprising/does not add enough new information to be interesting Inappropriate study design/methodological weaknesses/methods not described clearly enough Sample size too small/response rate too low Incorrect or unjustified conclusions Statistical analysis incorrect or inappropriate Wrong journal/readership; would not interest our readers Too badly written to understand
But sometimes something murky is lurking under the surface….
Murky things: misconduct • Fabrication: invention of data or cases • Falsification: wilful distortion of data including failure to report missing or negative data • Plagiarism: copying of ideas, data or words without attribution • Failing to get consent from an ethics committee for research and/or conducting research in humans without informed consent • Misreporting authorship contributions • Not revealing conflicts of interest
“How widespread is research and publication misconduct?”“How much does the peer review system catch?”
Some big cases (1) Jon Sudbø In January 2006 it was revealed that a paper by Norwegian researcher Jon Sudbø published in The Lancet in October 2005 was based upon fraudulent patient data. The article had suggested that some types of NSAIDs diminish the risk of oral cancer in smokers. However, it turned out that he had made up all the patient data; a quarter of all the subjects in The Lancet study had the same birthday!
Some big cases (2) Andrew Wakefield 1996 A paper by Wakefield and colleagues published in The Lancet suggested that there was a connection between the MMR vaccine and autism and colitis. 1998 No subsequent research was able to reproduce his findings. The paper was retracted. 2001 Immunisation rates dropped from 92% to 73%, almost certainly leading to a rise in mortality. 2010 Wakefield was struck off by GMC for over 30 charges of misconduct including four counts of dishonesty. Wakefield continues to defend his reputation.
Some big cases (3) Malcolm Pearce 1994: Obstetrician Malcolm Pearce claimed in BJOG to have successfully relocated an ectopic pregnancy, with a healthy subsequent birth. A junior doctor at St George’s blew the whistle. The patient had never existed.
“OK, so these are big cases.But does the little stuff really matter that much …?”
The journalology literature • 56% of newly appointed consultants reported witnessing research fraudGeggie (2001) J Med Ethics 27:344–6 • On average 2% of scientists admit to have falsified research at least once and up to 34% admit other questionable research practicesFanelli (2009). PLoS ONE 4(5): e5738 • 31% of submissions to a Chinese journal were plagiarized.Zhang. (2010) Nature 467:153 • 30% of submitted manuscripts included plagiarism from a previous publication of the senior author. 16% of submitted manuscripts included plagiarism from another investigator’s work and/or website.McFarlin et al (2010) IJ Exercise Sci (3):3 • 9.5% of authors in the BMJ were ‘guest authors’: i.e. they did not make any significant contribution to the paper Bates et al (2004) JAMA. 292(1):86-8.
How dangerous is it? • erodes public trust in the scientific community • is extremely costly to investigate and police • may create a flawed knowledge base (systematic reviews) • may negatively influence funders and politicians • may divert funding and effort away from reputable research • may influence the political process, esp in controversial areas e.g. stem cell research (Hwang Woo Suk) • can create false hope among patients PATIENTS COULD DIE
Corrections, Retractions and "Expressions of Concern" “The retraction or expression of concern, so labelled, should appear on a numbered page in a prominent section of the print journal as well as in the online version, be listed in the Table of Contents page, and include in its heading the title of the original article. The text of the retraction should explain why the article is being retracted and include a complete citation reference to that article.” Uniform Guidelines for manuscripts submitted to biomedical journals (The Vancouver Guidelines) http://www.icmje.org
But retracted papers live on … “Examination of one Nature paper by former Bell Labs physicist Jan Hendrik Schön, published in 2000 and retracted in 2003, revealed that it's been noted in research papers 17 times since, although the drop-off after retraction was steep: prior to being pulled, the paper was cited 153 times.” Unger K. Couzin J. Even Retracted Papers Endure. Science 2006;312(5770):40-41 “The difference between citation levels of flawed originals and corrected republications does not approach statistical significance until eight to twelve years post-republication.” Peterson GM. The effectiveness of the practice of correction and republication in the biomedical literature. J Med Libr Assoc. 2010; 98(2): 135–139.
Further information Brice, J. and J. Bligh. Author misconduct: not just the editors' responsibility. Medical Education, 2005. 39(1): p. 83-89 Smith, R. The trouble with medical journals. London: RSM Books 2006 Hames, I. Peer review and manuscript management in scientific journals: Guidelines for good practice. 2007, London: John Wiley and Sons. Wager, E., F. Godlee, and T. Jefferson. How to survive peer review. 2002, London: BMJ Books. Bligh, J. and J. Brice, New Year reflections on professionalism. Medical Education, 2005. 39(1): p. 2-3. Organisations: ICMJE: http://www.icmje.org/ World Association of Medical Editors: http://www.wame.org/ Committee on Publication Ethics: http://www.publicationethics.org/
