Scientific Reviewing for Physicians: a CLEAR Approach

1. John Iskander MD MPH CAPT, U.S. Public Health Service Senior Medical Consultant Office of the Associate Director for Science Centers for Disease Control and Prevention Scientific Reviewing for Physicians: a CLEAR Approach

2. “What do you read, my Lord?” “Words, words, words…” -Hamlet, William Shakespeare

3. Outline Importance of scientific reviewing Types of scientific reviews The CLEAR framework Categorizing comments Skill-building, resources, and training Questions

4. Importance of Scientific Review Essential to the scientific enterprise Building scientific credentials Author, reviewer, editorial board member Collaborating with colleagues Supporting science and the PHS mission Little or no formal instruction as part of medical training An underappreciated aspect of scientific leadership Not much in the “peer reviewed” literature on how to do a peer review!

5. USPHS Mission The mission of the U.S. Public Health Service Commissioned Corps is to protect, promote, and advance the health and safety of our Nation. As America's uniformed service of public health professionals, the Commissioned Corps achieves its mission through: Rapid and effective response to public health needs Leadership and excellence in public health practices Advancement of public health science

6. Basis for this Talk “Practice-based evidence” building on experience of serving as a peer reviewer, clearance reviewer, and policy reviewer OADS fellowship, scientific reviewing curriculum Short-term rotations support the professional development of a scientist interested in pursing a career track in the science leadership, management and policy field Discussions with and feedback from colleagues

7. Types of Scientific Review Subject matter expert (SME) Example: a peer review requested by a journal editor (or internally by a colleague) Is content scientifically valid, of importance, novel? Policy review Example: clearance or agency approval Is content consistent with policies and programs? Editorial review Example: Pre-publication review of an article or book chapter by editor Can clarity be improved without major changes to content?

8. Typical 3-Level Comment Framework Level 1 or mandatory Example: serious errors of fact or data interpretation, misstatement of agency recommendation or policy Level 2 or “revise or respond” These should be revised to improve understanding, but the authors can explain why revision is not possible (e.g. data to answer question is not available) Level 3 or editorial Often presented as inline edits If any such changes are mandatory, make sure this is clearly labeled

9. Before You Start Clearly understand: Your role, the type of document you are reviewing, and the type of review requested “Who are you, and what is this?” Respect the effort that went into creating the document Meet deadlines

10. The CLEAR Framework Clarity Logic Ethics “Agency” Relevance

11. What is Clarity? “Omit needless words.” -Strunk and White, The Elements of Style Shorter sentences, shorter (or no) parenthetical statements Consistent “voice” Do ideas presented conflict with each other or appear contradictory?

12. What is Logic? Flow of ideas Being able to follow an article’s “argument” Interpretation of findings that is in agreement with data presented Reasonable inferences drawn by authors about explanations for findings Should hold both within paragraphs and sections, between them, and for the document as a whole

13. What is Ethics? Not only whether there is documentation of IRB clearance and/or clinical trial registration, but also whether readers will perceive an ethical issue with the work that was done Vulnerable populations? Informed consent? Privacy/confidentiality concerns?

14. What is “Agency”? Dual meaning: Who is making the recommendations? Individuals, agency? In other words, who is the “agent”? Can the agency stand by the science and policy contained in the work? Of special importance for policy reviews

15. What is Relevance? “Information for action” What are the authors asking for-awareness of emerging or rare conditions, use of new diagnostic criteria or laboratory tests, reporting to public health authorities, some or all of these? Practical value to clinicians Rather than saying, “clinicians should look out for this incredibly rare disease” say “for patients with risk factors, this disease should be considered and the following laboratory tests may be obtained”

16. What Makes a Good Review? Feedback that is constructive, specific, and can be acted on by the author to improve some aspect of CLEAR Not: “This paragraph needs to be tightened up a little bit” Instead: “The second and third sentences can be eliminated as they duplicate information presented elsewhere”

17. What Makes a Good Review? A brief (2-3 sentence) summary of the review may be appropriate Might be only for your use, or may be shared with authors and/or editors Crystallizes your thinking, and points out “big picture” issues Do not rewrite extensively unless you are co-authoring or editing Recommending alternative language for a sentence or part of a sentence often appropriate (choose your words carefully-you may see them in print)

18. What Makes a Good Review? Be very clear about what must be changed, as opposed to what are recommendations for improvement The “three level” comment structure is one tool to facilitate this distinction Inline edits vs. marginal comments Communication With author, editor, and/or colleague

19. Tips and Tricks “Three paragraph” rule to see if document is ready for full review Nominalizations Strong and weak sentence parts Flipping back and forth-usually not a good sign Watch for “buried leads”-important ideas or statements that are hard to find within the document

20. Tips and Tricks For internal reviews, consulting with the author during review may be appropriate and helpful Reviewing and revising your own work-”sleep on it” Two stage review-read, then either re-read or review your comments Reviewing prior comments (yours or those of others) can be helpful Don’t repeat comments unnecessarily

21. Scientific Review Case Study

22. Review Case Study Document: 2012 Surgeon General's Report—Preventing Tobacco Use Among Youth and Young Adults (1395 pages, 7 chapters) Type of review: policy Timeframe for review: 2 weeks Methods: Multiple reviewers due to length of document Discussion with authors during review process to assure that highest priority comments were addressed Consolidation and adjudication of all comments If multiple reviewers involved, must “speak with one voice”

23. Resources for Review Medline/PubMed (www.ncbi.nlm.nih.gov/pubmed) Example use: does a reference really say what the authors assert they say? Often can be assessed by reviewing the abstract Clinical Trials registry (www.clinicaltrials.gov) Most journals require clinical trials to be registered here Agency or other medical libraries (and librarians)

24. Building Experience as a Reviewer How do you get to Carnegie Hall, or become a good reviewer? Practice, practice, practice Volunteer to review abstracts for COA or other scientific meetings Serve as a peer reviewer for journals, including Public Health Reports Review colleague’s manuscripts Write abstracts, reports, columns, journal articles, articles for COA Frontline etc.

25. What’s In It for Me? Some journals (e.g. Annals of Internal Medicine) offer continuing education (CE) credit to reviewers Some journals “credit” editors as part of the final publication In some agencies, scientific review may be a part of leadership responsibilities Associate Director for Science, Senior Scientist Building scientific and public health fund of knowledge, in or outside of your specialty

26. The Lighter Side of Being a Reviewer “Employment is a major cause of morbidity and mortality.” “A limitation of this survey is that it is prone to hat misclassification bias.” “We blindfolded reviewers…”

27. Acknowledgments Mary Ari James Stephens Rick Peavy LT Neelam Ghiya

28. Office of the Associate Director for Science