1 / 48

Beyond evidence based medicine

Beyond evidence based medicine. Fiona Godlee Editor, BMJ 3 rd European Conference on Scientific Publishing in Biomedicine and Medicine Leyden May 27 2010. Robert Merton The normative structure of science 1942. Communalism Universalism Disinterestedness Organised Skepticism.

tomai
Download Presentation

Beyond evidence based medicine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Beyond evidence based medicine Fiona Godlee Editor, BMJ 3rd European Conference on Scientific Publishing in Biomedicine and Medicine Leyden May 27 2010

  2. Robert Merton The normative structure of science 1942 Communalism Universalism Disinterestedness Organised Skepticism

  3. A comment on the state of the medical literature from Drummond Rennie, deputy editor, JAMA There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.

  4. McGauran et al. Trials 2010

  5. GI outcomes, VIGOR, NEJM 2000 17/1000

  6. Placebo vs Rofecoxib or Celecoxib: comparison of MI rates JAMA 2001

  7. Thrombotic events, Approve, NEJM 2004

  8. BMJ 2005

  9. Publication bias Selective publication of results, so that positive results are over represented and negative ones under represented

  10. Publication bias in medical research Positive studies are more likely to be: • Published Stern and Simes, BMJ 1997 • Published faster Ioannides, JAMA 1998 • Published in higher impact journals Easterbrook et al, Lancet 1991: Tierney and Stewart, 1997 • Cited old style narrative review articles

  11. Publication bias - Who’s to blame? • Editors/peer reviewers • Mahoney 1977 • Epstein 1990 • Researchers • Dickersin et al, Controlled Clinical Trials 1987 • Olson et al, JAMA 2002

  12. Publication bias linked to industry funding • Rochon et al. Arch Intern Med 1994 • Lexchin, Bero et al. BMJ 2003

  13. Duplicate publication and salami slicing Trials of odansetron • 84 trials that included information on 11 980 patients • In reality only 70 trials and 8645 patients (17% of the studies had been published more than once and the number of patients had been inflated by 28%) • Impossible to tell from published studies • Four pairs of identical trials were published by completely different authors without any common authorship (Misconduct) (Tramer et al, BMJ 1997: 315: 635-640)

  14. Effectiveness of odansetron(Tramer et al, BMJ 1997: 315: 635-640)

  15. Some things that bother editors • Bias • Data manipulation/suppression • Duplicate publication • Fabrication • Falsification • Gift/ghost authorship • Plagiarism • Self delusion • Undeclared conflict of interest • Wrong observations/analysis/references

  16. Errors in good faith Trimming and cooking Fraud Bias Data manipulation/suppression Duplicate publication Fabrication Falsification Gift/ghost authorship Plagiarism Self delusion Undeclared conflict of interest Wrong observations/analysis/references Stephen Lock, BMJ

  17. Errors in good faith Trimming and cooking Fraud Manipulating data Suppressing inconvenient facts Fabrication Falsification Plagiarism Wrong observations Wrong analysis Wrong references Bias Self delusion Gift authorship Duplicate publication Salami publication Undeclared conflicts of interest Stephen Lock, BMJ

  18. Undeclared competing interest/ghosts and guests • RCT of Vioxx versus Naproxen Lisse et al. Ann Intern Med 2003; 139: 539-46 • New York Times,24 April 2005 “Merck designed the trial, paid for the trial, ran the trial…Merck came to me after the study was completed and said, ‘We want your help to work on the paper.’ The initial paper was written at Merck, and then sent to me for editing.”

  19. Guests and ghosts are not “a thing of the past” • Surveyed authors of 900 articles published in high impact general medical journals in 2008 • 1 in 5 respondents admitted to at least one guest author (no change since 1996) • Nearly 8% admitted to at least one ghost author (slightly lower than 1996) (Flanagin A, et al. Presented at Peer Review Congress, 2009)

  20. Influence of financial links with industry on authors of reviews and letters • Stelfox et al, NEJM 1998 • Lexchin, Bero et al. BMJ 2003 • Barnes DE, Bero LA. JAMA 1998 • Yank, Rennie, Bero, BMJ 2007 • Wang et al, BMJ 2010

  21. Effect of conflict of interest statements on readers’ perceptions • Two RCTs Chaudry et al, BMJ 2002 Schroter et al, BMJ 2004 • A research article with a financial conflict of interest was rated less highly by readers than the same paper with no COI or non-financial COI

  22. What is it about conflicts of interest that bother us? • Having them at all? • Failing to declare them? • Is disclosure enough?

  23. How do we act on disclosed information? Three options • Disclosure is enough in itself. After that, anything goes, caveat emptor • Disclosure is followed by a judgement about the degree of potential conflict • Disclosure of a conflict of interest is a bar to taking part (membership of a panel, authorship of an editorial, involvement in a piece of research)

  24. Science depends on… • Open critique and debate • Correction/retraction • Honesty and transparency • Identification and minimisation of bias • Repeatablity • Refutation or confirmation • Viewed as a public good • Ethics

  25. Why does integrity matter? Damage to the integrity of the biomedical literature brings: • Pursuit of wrong avenues of scientific enquiry • Participants and patients put at risk • Waste of time and resources • Loss of public trust in science • Less willingness to fund research • Less willingness among patients and the public to take part in research

  26. Sources of pressure on integrity • Science is a human enterprise • Human beings are fallible • Careers and huge sums of money are at • Libel laws increasingly being invoked, protecting those with money • Science is based on a system of trust • Existing checks and balances are only partially effective • Training and supervision are inadequate • The biomedical literature is fragmented, making it hard to get to the “truth” With the result that a lot of published research is of poor quality, and some of it (how much?) is fraudulent

  27. “The community presents a natural work setting for an editor, not the laboratories or surgeries. The editor is a community worker and a teacher.” Andrija Stampar, one of the founders of WHO

  28. Problems with peer review • Slow • Expensive • Biased • Unaccountable • Stifles innovation • Bad at detecting error • Hopeless at detecting fraud • And journals have conflicts of interest too

  29. How do journals make their money?

  30. “Radical transparency” • Nearly all decision making is carried publicly • Not only the ordinary information of interest to the community is made freely available, but all (or nearly all) meta-levels of organizing and decision making are also published • Includes draft documents, all arguments for and against a proposal, the decisions about the decision making process itself, and all final decisions • Excludes data relating to personal security, passwords etc (from Wikipedia)

  31. Moving towards greater transparency • Trial registration • Protocols • Data sharing • Mandate data availability • Unlimited space online • Uniform COI form • Open access • Open peer review/pre-publication history • Article level metrics

  32. But will any of this really help? A more radical future • Stop using opinion leaders with any financial conflicts of interest? • Stop pharmaceutical companies directly evaluating their own products? • An end to pre-publication peer review? • Publication of entire data sets

  33. Why we need raw data Godlee F, Clarke M Why don’t we have all the evidence on oseltamivir? BMJ 2009

  34. Conclusions • The integrity of the biomedical literature is essential if we are to practice evidence based medicine • Loss of integrity distorts the scientific record and threatens public trust in science, funding and participation in research, and patient safety • The threats to integrity are many and various • Some threats are caused by honest error, but a significant proportion are intentional, for commercial or career gain (misconduct/fraud) • Additional threats come from unnecessary and poorly done research

  35. Conclusions • Current efforts to protect the integrity of medical knowledge are inadequate • Some hope lies in better education, greater transparency, and ever more rigorous peer review • Tackling the influence of the drug industry remains a major challenge for clinicians, researchers, journals, journalists, and the public • More radical alternative forms of research funding and data publication are needed

  36. Thank you fgodlee@bmj.com

More Related