Reviewing Manuscripts Submitted to Scientific Journals

1. Bruce M. Damon, PhD Institute of Imaging Science Depts. of Radiology and Radiological Sciences, Biomedical Engineering, and Molecular Physiology and Biophysics and Program in Chemical and Physical Biology Vanderbilt University, Nashville TN Reviewing Manuscripts Submitted to Scientific Journals

2. Career Development Series Previous presentations • Giving Effective Oral Presentations about Science • Finding a Job in Academia • Professional Expectations • Federal Funding 101 • Grant and Proposal Development Mini-Course

3. Learning Objectives Overall Objective • Learn the purposes, procedures, and ethical issues associated with the scientific peer review process Specific Objectives • Obtain a broad perspective of the peer review process • Learn an approach to reviewing manuscripts • Learn some ethical issues associated with the peer review process

4. Presentation Overview Introduction to the Peer Review Process • Purposes • Why participate in peer review? • Overview of the process An Approach to Manuscript Review Ethics of the Peer Review Process

5. Purposes of Peer Review Peer review exists in order to evaluate the: • Originality • Significance • Scientific content • Ethical aspects • Presentation • Appropriateness to the journal in question of manuscripts submitted for publication. Is one of the purposes of peer review to make the manuscript better? Introduction How-to Guide Ethics

6. Why Would I Want to Do This? Participating in the peer review process is both a privilege and a responsibility. • Peer review is a critical part of the scientific process and helps to ensure that high quality work is published. • Be a good citizen • Personal benefits: • Insight into manuscript review process • Establishes your credentials • Stay abreast of the latest work • Doing a good job on your reviews tends to be contagious Introduction How-to Guide Ethics

7. Overview of the Process Manuscript submitted Assigned to Assoc. Editor Assoc. Editor recruits reviewers Reviewers evaluate manuscript Editorial decision Revise and resubmit Reject Accept Introduction How-to Guide Ethics

8. Overview of the Process Manuscript re-submitted Examined by Assoc. Editor Reviewers evaluate manuscript Editorial decision Revise and resubmit Reject Accept Introduction How-to Guide Ethics

9. Overview of the Process Manuscript re-submitted Examined by Assoc. Editor Editorial decision Revise and resubmit Reject Accept Introduction How-to Guide Ethics

10. Overview of the Process Manuscript submitted Assigned to Assoc. Editor Assoc. Editor recruits reviewers Reviewers evaluate manuscript Editorial decision Revise and resubmit Reject Accept Introduction How-to Guide Ethics

11. Presentation Overview Introduction to the Peer Review Process An Approach to Manuscript Review • Overall workflow • Each step of the review process Ethics of the Peer Review Process

12. Overall Workflow Receive, accept invitation First reading Think about it, do some homework Detailedreading Perform review Make recommendation Introduction How-to Guide Ethics

13. Receiving and Accepting the Invitation How does one get to become (and remain) a reviewer? • Publish a sufficient body of work within an area to be identifiable as an expert • Make yourself known to editors: interact with them personally; make a positive impression at conferences • Editors may consider your objectivity, helpfulness, timeliness, comprehensiveness, and clarity when selecting reviewers in the future Introduction How-to Guide Ethics

14. Receiving and Accepting the Invitation Dear Bruce Manuscript # DRWHO-07-05 entitled “The Angels Take Manhattan" has been submitted to the journal Diagnostic Radiology in Welsh Healthcare Organizations. As an acknowledged expert in the topic covered by the paper we would welcome a review of this manuscript from you. The abstract appears at the end of this letter, along with the names of the authors. Please let me know within 5 days if you will be able to review this paper. If you are unable to review this paper, would you take a moment to please recommend one or two other possible reviewers with expertise in this area. If you do choose to review, we will contact you via email with instructions for accessing our online manuscript submission and review system. You will then have access to the manuscript and reviewer instructions in your Reviewer Centre. I would ask that you complete your review within 2 weeks. Introduction How-to Guide Ethics

15. Receiving and Accepting the Invitation Please click the link below to agree to review this paper and to be given your access details. If you are unable to review we would appreciate it if you could suggest one or more alternative reviewers. Agreed: http://mc.manuscriptcentral.com/drwho?URL_MASK=tardisyes Declined: http://mc.manuscriptcentral.com/drwho?URL_MASK=tardisno Unavailable: http://mc.manuscriptcentral.com/drwho?URL_MASK=tardisun Sincerely, David Tennant Associate Editor Introduction How-to Guide Ethics

16. Receiving and Accepting the Invitation MANUSCRIPT DETAILS TITLE: The Angels Take Manhattan AUTHORS: A. Pond, R. Williams, R. Song, I.M. Nerdy, M. Smith. ABSTRACT: Amy, Rory, River, and the Doctor encounter aliens that take the form of statues while they are being observed, but can otherwise move at exceedingly high rates. These aliens feed by sending people back in time and consuming the resulting release of potential energy. If you see one, don’t blink. Also, you might do well to avoid Liberty Island. Introduction How-to Guide Ethics

17. Receiving and Accepting the Invitation Should I accept this invitation to review? • Am I sufficiently expert in this area? • Do I have the time and inclination to do this in a proper and timely way? • Do I have a conflict of interest? • Existence of a contentious dispute with author • Close personal or professional relationship with author • If for any reason you feel that you might be biased • Editors would prefer to find an alternative reviewer than deal with inexpert, late, unfair, or poorly done reviews Introduction How-to Guide Ethics

18. Overall Process and Workflow Receive, accept invitation First reading Think about it, do some homework Detailedreading Perform review Make recommendation Introduction How-to Guide Ethics

19. First Reading Goals for the first reading: • Get the big picture • Overall goal • Approach • Major findings • Major conclusion • Find out what one needs to look up or learn more about • Do think ahead to the items you’ll be asked to comment on, but remain open-minded Introduction How-to Guide Ethics

20. Overall Process and Workflow Receive, accept invitation First reading Think about it, do some homework Detailedreading Perform review Make recommendation Introduction How-to Guide Ethics

21. Think About It, Do Some Homework You might find that you need to: • Learn more about a particular topic • Consult a colleague, maintaining confidentiality • Read some literature or a textbook • Go back and do a second “big picture” level of review • Give it a day or so to percolate – make sure you have a good overall sense of what the paper is about Introduction How-to Guide Ethics

22. Overall Process and Workflow Receive, accept invitation First reading Think about it, do some homework Detailed reading Perform review Make recommendation Introduction How-to Guide Ethics

23. Detailed Reading General thoughts • Much like reading a published paper • But with some differences and additions: • In more detail; takes more time • Level and nature of critical analysis differ • You are wearing two hats: Journal, scientific process Author Introduction How-to Guide Ethics

24. Detailed Reading Originality • Actual originality and level of significant expansion of the existing body of knowledge • Accuracy with which these are communicated Significance • Presuming that the manuscript’s weaknesses can be fixed, what would its impact be? Introduction How-to Guide Ethics

25. Detailed Reading Scientific content: Title • Does the title accurately and succinctly communicate the major contribution or finding of the paper? Scientific content: Abstract • Is the abstract written clearly and succinctly? • Is the abstract accurate and consistent with the body of the manuscript? Introduction How-to Guide Ethics

26. Detailed Reading Scientific content: Introduction • Does the introduction succinctly and accurately review all of the relevant literature (and only the relevant literature)? • Does it establish and defend the significance of the study? • Does it establish, clearly state, and support the purpose(s) for conducting the study? • Is the stated purpose consistent with the rest of the paper? Introduction How-to Guide Ethics

27. Detailed Reading Scientific content: Methods • Experimental design • Appropriateness of the sample selected • Theoretical soundness of the design: • Were the appropriate control groups used? • Are there confounding variables? Introduction How-to Guide Ethics

28. Detailed Reading Scientific content: Methods • The specific methods • Applies to data acquisition, data analysis, statistics/ hypothesis testing • Were good practices used? • Are the reasons for each method’s selection either obvious or well defended? • Are the methods described clearly, comprehensively, and in sufficient detail?  Introduction How-to Guide Ethics

29. Detailed Reading Scientific content: Methods • Is there a clear relationship among the purpose, methods, results, and conclusions? • When considering the methods, realize that • Errors of data analysis and statistics can be fixed • Errors of experimental design and data acquisition either can’t be fixed or will take a lot more work • Either way, the outcome may be uncertain Introduction How-to Guide Ethics

30. Detailed Reading Scientific content: Results • Are the data presented clearly and with the appropriate • Units • Statistics • Mechanism of presentation (text, table, and figures) • Is there unnecessary redundancy? • Does the text appropriately introduce and discuss the figures and tables? Introduction How-to Guide Ethics

31. Detailed Reading Scientific content: Results • Figures and tables • Is each one necessary? • Presentation: size, resolution, aspect ratio, use of color • Are axis scales and labels, captions, legends, etc complete, accurate, and appropriate? • A figure or table should be able to stand on its own Introduction How-to Guide Ethics

32. Detailed Reading Scientific content: Results • What is your impression of the data? • Do they make sense? • Consider the variations due to experimental treatments vs. measurement error or individual subject variability • Are there any aspects of the data that stand out as unusual? • Are the data that are presented consistent with the Methods and Discussion sections? Introduction How-to Guide Ethics

33. Detailed Reading Scientific content: Discussion and Conclusions • Are the major findings summarized and highlighted? • Is the existing scientific literature fully and accurately considered? • Interpretations and conclusions: • Are they supported by the experimental design and the data? • Are all alternative explanations considered? • Are the limitations of the study acknowledged and considered? • Are the significance and impact of the results and conclusions discussed? Introduction How-to Guide Ethics

34. Detailed Reading Ethical aspects • Statements of ethics committee approvals • Evidence of scientific misconduct • Plagiarism • Duplicate publication • Unreported apparent or actual conflict of interest of the authors • Reporting such concerns is unpleasant and difficult, but it is the right thing to do and an essential responsibility of a reviewer. Introduction How-to Guide Ethics

35. Detailed Reading Presentation • We’ve talked about some of this already • Overall organization, economy of expression, and clarity • English language usage • Does the paper need linguistic review by a native English speaker? • Distinguish matters of style from matters of correct usage or scientific ambiguity • Attention to detail Appropriateness to the journal in question Introduction How-to Guide Ethics

36. Overall Process and Workflow Receive, accept invitation First reading Think about it, do some homework Detailedreading Perform review Make recommendation Introduction How-to Guide Ethics

37. Perform your Review The journal will ask some standard questions Example: Magnetic Resonance in Medicine Is the manuscript in the appropriate category? (Y/N) Originality of content (1-5) Clarity of presentation(1-5) Quality of organization(1-5) Quality of figures and tables(1-5) Significance of results(1-5) Soundness of conclusions(1-5) Style, grammar and syntax(1-5) Accuracy (are there substantive errors?)(Y/N) Introduction How-to Guide Ethics

38. Perform your Review The journal will ask some standard questions Needs linguistic editing by a native English speaker? (Y/N) Does paper conform with the MRM Style Guide? (Y/N) Is the scope of the references complete? (Y/N) Can manuscript be shortened without compromising intended message? (Y/N) Is the abstract appropriate and complete? (Y/N) Importance (potential implications on other work, reader interest) (1-5) Quality (creativity, quality of methodology, data, presentation) (1-5) Do you recommend publication? (Y/N/Minor or Major) Is MRM the appropriate journal? (Y/N) Introduction How-to Guide Ethics

39. Perform your Review Written comments to author General things to do • Be the sort of reviewer whom you would like to have: • Critical • But also, be constructive, collegial, fair, thorough, detailed, specific, accurate, knowledgeable, thoughtful • Identify and consider both the strengths and weaknesses of the manuscript • Ensure that the written comments match the scores Introduction How-to Guide Ethics

40. Perform your Review Written comments to author General things not to do • Do not to rewrite their paper or redesign their study to match your own experimental aims • Do not communicate a recommendation regarding publication directly to the author • Do not communicate ethical concerns directly to the author Introduction How-to Guide Ethics

41. Perform your Review Written comments to author • Start with a paragraph that summarizes the paper, lists major strengths and weaknesses, and states the overall impact or impression • Then, list specific points of critique (often divided into separate sections for major and minor concerns): • Page 14, paragraph 2, lines 2-4: There may be another explanation for this finding, namely that…See the paper by Joe Bagadonuts (MRM, 57:752, 2007). The reviewer recommends including this alternative explanation also. Introduction How-to Guide Ethics

42. Overall Process and Workflow Receive, accept invitation First reading Think about it, do some homework Detailedreading Perform review Make recommendation Introduction How-to Guide Ethics

43. Make your Recommendation General approach • Make or break issues: • Are the methodological weaknesses related to experimental design or data acquisition, or are they related to data analysis? • Presuming that the manuscript’s weaknesses can be fixed, what would the impact of the work be? • Appropriateness of the manuscript for the journal Introduction How-to Guide Ethics

44. Make your Recommendation General approach • Integrate and analyze the other points of strength and weakness • Originality and innovation • Potential significance • Scientific quality • Appropriateness of conclusions • Quality of presentation • Weigh all of these factors and make your decision: Accept, Minor Revisions, Major Revisions, Reject • Wait a day and reread the review before submission Introduction How-to Guide Ethics

45. Make your Recommendation Confidential comments to editor • Typically a paragraph that • Summarizes the paper • Lists major strengths and weaknesses • Synthesizes and analyzes these factors to form an overall impression and to support your overall recommendation • This is the place to express ethical concerns Introduction How-to Guide Ethics

46. After the Review is Submitted Editorial decision • Blind-copied to all reviewers • Good opportunity to compare your review to those of other reviewers Introduction How-to Guide Ethics

47. After the Review is Submitted Analyzing author responses For each point of critique: • Page 14, paragraph 2, lines 2-4: There may be another explanation for this finding, namely that… See the paper by Joe Bagadonuts (MRM, 57:752, 2007). The reviewer recommends including this alternative explanation also. Introduction How-to Guide Ethics

48. After the Review is Submitted Analyzing author responses There should be a response: The reviewer is correct. We have added this discussion to page 15, paragraph 1, lines 2-4. We appreciate this comment from the reviewer, but we respectfully disagree. We do not feel that this alternative explanation applies in this context, because... We have added this point to the Discussion on page 15, paragraph 1, lines 2-4. Introduction How-to Guide Ethics

49. Presentation Overview Introduction to the Peer Review Process An Approach to Manuscript Review Ethics of the Peer Review Process

50. Ethical Issues Related to Peer Review Things we’ve already covered • Reviewer and author conflicts of interest • Timely review • Concerns related to ethics committee approval • Identifying and reporting plagiarism or other scientific misconduct Other stuff… • Plagiarism by the reviewer • Confidentiality of the review process Introduction How-to Guide Ethics