FIRST AID FOR THE PEER REVIEW PROCESS

1. FIRST AID FOR THE PEER REVIEW PROCESS Giuseppe Biondi Zoccai DivisionofCardiology, UniversityofTurin, Turin, Italy Meta-analysis and Evidence-based medicine Training in Cardiology(METCARDIO), Ospedaletti, Italy How to develop effective abstracts and manuscripts in interventional cardiology GISE 2009 – 22 October 2009 – 13.25-13.50

2. LEARNING GOALS • What is peer review, and why have we to survive it through? • What should you do and not do when actually peer reviewing other colleagues’ works? • What should you do and not do when surviving through peer review? • Case studies

3. WHY AM I GIVING YOU THIS LECTURE? Peer reviews I have completed since 2003 expected

4. LEARNING GOALS • What is peer review, and why have we to survive it through? • What should you do and not do when actually peer reviewing other colleagues’ works? • What should you do and not do when surviving through peer review? • Case studies

5. DEFINITION Peer vb intr. 1. to look intently with or as if with difficulty. 2. to appear partially or dimly. n. 1. a person who is an equal in social standing, rank, age, etc. Collins Dictionary of the English Language, London & Glasgow: Collins, 1979

6. ORIGINS OF PEER REVIEW • Peer review was born in the 17th century in the UK, likely thanks to Henry Oldenburg, the founder of Philosophical Abstractions (1665) • He originally introduced the practice of soliciting opionions on manuscripts from more knowledgeable external colleagues

7. NOTABLE EXCEPTIONS • Not all manuscripts undergo thorough peer review. Notable examples? • Albert Einstein’s “AnnusMirabilis” papers, published in 1905 in AnnalenderPhysikbyMax Planck(fatherof quantum theory and Nobel prizewinner), whoread the papers and decidedaltogethertopublishthem • more humbly…Biondi-Zoccaiet al, Int J Cardiol2005;100:119-23

8. WHAT’S WRONG WITH PEER REVIEW? • Unreliable • Unfair • Fails to truly validate or authenticate • Unstandardized • Idiosyncratic • Open to every sort of bias

9. WHAT’S EVEN WORSE? • Stifles innovation • Perpetuates the status quo • Rewards the prominent but punishes the weak • Unnecessarily delays dissemination • Very expensive • Insufficiently tested

10. YES, BUT… • It has been said that democracy is the worst form of government except all those other forms that have been tried from time to time – W. Churchill • The sameappliestopeerreview

11. WHAT ARE ITS PROS? • Filters bad from good, and better from less good, as well as possibly identifying misleading and false research • Protects patients • Guides authors to improve the quality of their article, and improve their whole research approach • Authenticates work, assuring quality • Improves readability • Broadens participation and dialogue

12. PARAPHRASING GROUCHO I would never enter into a club that would accept me as a member… Groucho Marx, 1980-1977

13. LEARNING GOALS • What is peer review, and why have we to survive it through? • What should you do and not do when actually peer reviewing other colleagues’ works? • What should you do and not do when surviving through peer review? • Case studies

14. ARE PEER REVIEWERS UNBIASED? Mahoneyet al, CTR 1977

15. WILL PEER REVIEW LET YOU SLEEP? On average a review will take you around 3 hours (actually I usually complete one in 45’) McNuttet al, JAMA 1990

16. SHOULD YOU BE BLINDFOLDED? Godleeet al, JAMA 1998

17. WHO ARE THE BEST REVIEWERS? • Other (also weak) predictors: • Coming from good institutions • Known to the editors • Had methodological training (statistics & epidemiology) Klieveret al, AJR 2005

18. WHO ARE THE WORST REVIEWERS? Italian peer reviewers don’t like Italian manuscripts! Opthofet al, CardiovascRes 2002

19. THREE PIVOTAL QUESTIONS FOR PEER REVIEW • Do I understand it? Are the question and the methods clearly explained? • Do I believe it? Are the conclusions justified by the data and are the methods valid? • Do I care? Is the question important and interesting? Wageret al, Howtosurvivepeerreview. BMJ Books 2002

20. CHECKLIST FOR PEER REVIEWERS: ISSUES FOR COMMENT • Importance of research question • Originality of work • Delineation of strengths and weaknesses of methodology/experimental/statistical approach/interpretation of results • Writing style and figure/table presentation • Ethical concerns (animal/human) Benoset al, AdvanPhysiolEduc 2003

21. SUBTLETIES OF PEER REVIEW: PRIORITY – i.e. how to kill a paper • Priority means novelty, originality, and likelihood of generating interest, irrespective from quality, validity, and methodology • Manuscript can be judged as low, mid, high, or top priority • Some journals (e.g. the Journal of Cardiovascular Medicine) use scores, e.g. ranging from 0 to 100, with 100 meaning top priority

22. SUBTLETIES OF PEER REVIEW: DECISION • Decision means recommending a specific editorial handling of the manuscript, and can be distinguished in: • accept as is • accept/reconsider after minor revisions • accept/reconsider after major revisions • reject but reconsider on a de novo basis • reject

23. SUBTLETIES OF PEER REVIEW: DECISION • Accept as is: it can be published as it stands • Accept after minor revisions: it can be published EVEN IF not all my comments are taken into account • Accept/reconsider after major revisions: it can be published ONLY IF all my comments are taken into account • Reject but reconsider on a de novo basis: it must be changed altogether, and priority also reappraised after resubmission • Reject: just send it back, it ain’t worth it

24. EXAMPLE OF ACTIVE PEER REVIEW

25. LEARNING GOALS • What is peer review, and why have we to survive it through? • What should you do and not do when actually peer reviewing other colleagues’ works? • What should you do and not do when surviving through peer review? • Case studies

26. FIRST, DON’T PANIC!

27. SHOULD YOU TRUST PEER REVIEWERS? Opthofet al, CardiovascRes 2002

28. IDENTIFYING THE MOST APPROPRIATE TARGET Wageret al, Howtosurvivepeerreview. BMJ Books 2002

29. COVER LETTER

30. REPLYING TO REVIEWERS COMMENTS • Do not ignore comments, but make most if not all of changes • Be calm, objective, and polite even if reviewer’s comments were harsh • Deconstruct each of the messages into individual items • Respond to each item thoughtfully • Make responses clear

31. REPLYING TO REVIEWERS COMMENTS • Don’t have to fully accept suggestions but must give reasons that will convince editor your opinion is reasonable • Be pragmatic and not dismissive of reviewer’s work • Explain just enough to enable you to survive • Benefit from it, and learn also how to become a competent reviewer

32. REBUTTAL LETTER

33. AFTER RESUBMISSION OF REVISION • No guarantee will be published • Editor will consider new version and your replies to comments • Editorial process can be subjective, and sometimes downgrade priority -> rejection • Reviewer’s comments only one factor • Editor may reject paper even if reviewer’s comments were minor • Editor has absolute discretion

34. IF REJECTED CAN APPEAL • If you think reviewer’s overlooked or misunderstood something important • Appeal by writing a letter stating your case –> appeal letter • Rare decision overturned but it does happen • If appealing—send new copy of paper—rejected papers do not remain on file (i.e. de novo submission)

35. WHAT TO DO AFTER REJECTION • Reviewers critical of basic methods—may need to rethink study and do further data analysis • Reviewers critical on style and presentation—fix problems before resubmitting to another journal • Three repeat rejections—completely reassess entire approach or search for appropriate target

36. WHEN ACCEPTED • Receive page proofs—typeset copy of work—how looks in journal • May take several months to receive • Time for final check • Journal usually send proof reading instructions you must follow • Usually standard proofreading marks

37. TYPICAL REASONS FOR FINAL REJECTION • Fundamentally weak hypothesis • Lack of clinical relevance • Old knowledge with no new or useful material • Two or three of reasons 1–3 • Reasonable text, but images are of very poor quality, are inappropriate, or are incorrectly interpreted • Too many methodologic errors • Hypothesis adequate, but poor study design, methodology, or statistics • Lacking in logic; initial premise not logically supported by methods and results Eharaet al, AJR 2007

38. TYPICAL REASONS FOR FINAL REJECTION (CONT.) • None of the other reasons, but reviewers do not like the article • Previously published • Sample population too small or biased to justify results and conclusion • Well written but better suited for another journal • Major language problems; English not primary language of author • Too poorly written, phrased, or presented • Failure to follow journal guidelines • Lack of correlation between purpose and results • Poor statistics, beyond salvage Eharaet al, AJR 2007

39. TEN TIPS TO FOR SURVIVAL • Properly organize the manuscript • Clearly state the study question and rationale • Explain methods in a systematic fashion • Structure methods and results in a similar manner • Make the discussion session concise • Explain if – and why – your study results are important • Avoid overinterpreting • Explain the limitations • Account for unexpected findings • Fully incorporate reviewers’ suggestions Provenzale et al, AJR 2007

40. LEARNING GOALS • What is peer review, and why have we to survive it through? • What should you do and not do when actually peer reviewing other colleagues’ works? • What should you do and not do when surviving through peer review? • Case studies

41. THE LOVERBOY REVIEWER Lotrionteet al, Am J Cardiol 2008

42. THE LOVERBOY REVIEWER Lotrionteet al, Am J Cardiol 2008

43. THE INQUIRING REVIEWER Biondi-Zoccaiet al, Am Heart J 2008

44. THE INQUIRING REVIEWER Biondi-Zoccaiet al, Am Heart J 2008

45. THE DELUSIONAL REVIEWER Sheibanet al, J Am CollCardiol 2009

46. THE DELUSIONAL REVIEWER Sheibanet al, J Am CollCardiol 2009

47. THE NASTY REVIEWER Sheibanet al, J IntervenCardiol 2008

48. THE NASTY REVIEWER Sheibanet al, J IntervenCardiol 2008

49. THE PESSIMISTIC REVIEWER Biondi-Zoccaiet al, Am Heart J 2008

50. THE PESSIMISTIC REVIEWER Biondi-Zoccaiet al, Am Heart J 2008