AJRCCM Early Career Researchers’ Group How to Be a Successful and Effective Peer Reviewer

1. AJRCCM Early Career Researchers’ Group How to Be a Successful and Effective Peer Reviewer 21st May 2018 James Allinson PhD

2. This Session • What is the peer review process? • How can I be an effective peer reviewer? ……and why should I want to be one? • Early Career Research Group: Opportunities at the AJRCCM

3. AJRCCM Peer Review Process Image Submission Manuscript Returned Basic Criteria Editor/Deputy Editor Direct Rejection Associate Editor Reviewers (2+) Rejection (+ Comments) Revision Major/Minor Editorial Decision Acceptance

4. Peer Review Process: Aims and Motives AJRCCM Editors: • To publish the highest-quality original articles in respiratory, critical care, and sleep medicine • To advance science/medicine and be educational • Publications reflecting spectrum of sub-specialties • To decide which manuscripts should be prioritized for publication Reviewers: • To advise on scientific quality, data quality, and originality • Offer specialist expertise

5. Involvement in Peer Review • Duty to the scientific archive and community • Fun • Educational • Ambition

6. Becoming an Effective Peer Reviewer • First-hand • Mentorship • Educational Resources

7. Becoming an Effective Peer Reviewer Hoppin, FG. AJRCCM 2002

8. Reviewer Attitude • Approach with a helpful attitude: • Review and improve quality of papers • Stakes are high for authors • Patience, objectivity, and openness • Good reviews take effort and time with limited immediate reward • Rein in competitive instincts

9. Invitation to Review – Should I Accept? Consider….. • Is the topic of interest to you? • Is the topic within your expertise? • Are you able to commit time to review • Are there any relevant conflicts of interest

10. First Read • Individual approach to reading a paper • Active approach – notations/questions • Abstract and conclusion • What do the authors think is new/important • Read full paper • Problems with science • Problems with ethics • Problems with presentation

11. First Read Problems with science • Contradictions/circular reasoning • Unjustified conclusions/causation/extrapolation • Trivial question • Statistics • Inappropriate tests • Claims not supported by tests

12. First Read Problems with ethics • Fraud/Misconduct (Fabrication/Falsification/Plagiarism) • Duplication/Redundancy • Consent/Confidentiality • Poor records/Image manipulation/Authorship disputes/Protocol deviations • Approvals

13. First Read Problems with the presentation • Redundancies, irrelevancies, and unnecessary excursions lead to boredom and distraction • Abbreviations: Defined, Standard, vs Non-Standard • Jargon creating confusion • Language barrier • Poor focus • Thrust of a paragraph should be clear at the beginning • Style

14. Big Questions • What is new? • What have the authors missed? • Have the authors acknowledged other reasonable hypotheses? • Have they considered methodological limitations? • Does their discussion address all discrepancies or agreements between their results and those of other researchers? Requires awareness of literature and the ability to apply and relate scientific principles and findings to new science

15. But…. Senior reviewers reportedly do worse job than juniors • Spend less time • Cut review short if major issue so don’t detail all problems

16. Pause for Reflection Set aside for a few days (maybe keep thinking)

17. Second Read • Re-read article • Review and revise your initial impressions and comments • Integrity of science, Quality of reasoning, Application of scientific principles and knowledge • NOVELTY of idea/conclusion/data/methods • Presentation: clarity, precision, or completeness NB…….If you have difficulty understanding then so will reader

18. Making Your Decision • What do you think the ultimate outcome should be? • Is there anything that need to happen before then? • Uncertainty: e.g. compelling question/intriguing idea but science is weak • Convey how to improve science • Outline pros and cons to AE • Perhaps convey dilemma

19. Comments to the Editors • Confidential comments to the editors • Comments to the authors

20. Confidential Comments to the Editors • Summary (3 or 4 sentences). Identify topic of study, indicate basic approach, main findings, paraphrase main conclusions • List main criticisms/questions in descending order of importance • Summarize basis, indicate importance and remedy • Indicate and characterize recommendations and degree of confidence in recommendations • e.g. “this is a novel idea, worth inviting major revision” • e.g. “I suspect authors will have difficulty answering question satisfactorily”

21. Comments to the Authors • Observations and analysis must be clear to authors • Dispassionate: avoid expressing emotion/severe disapproval/harsh criticism • Help authors understand how article may be improved • Show authors what reader takes from paper so helps them focus and prepare responses • If any aspects not evaluated e.g. complex stats state explicitly

22. Comments to the Authors • General, Major, Minor (numbered) • Each criticism/question must be explained • Never unsupported statements eg “inadequate controls”: Must be a “because….” • Occasionally indicate alternative approaches • Presentation advice • Good science should be published to help authors clarify ideas • Tell authors what you understand from what they have written and tell them where you got lost

23. Major Tips • Do not rush • Repeated reading • Look for NOVELTY and GOOD SCIENCE • Deliver findings clearly and courteously

24. How Do I Become a Reviewer? Wedzicha et al. AJRCCM 2015

25. Editors in Training

26. AJRCCM Early Career Group Remit

27. Mentorship Editors in Training Junior Editors in Training Junior Reviewers

28. Conclusions • Effective peer review is essential to maintaining the academic archive • Being a peer reviewer is a privilege with obligations to both your peers and to science • Rewarding endeavour • The Blue Journal Early Career Group offers opportunities to become involved in the peer review process