Interpretation of Reviewer Comments

1. Interpretation of Reviewer Comments The resubmission gamble!

2. The Peer Review Process Dual Review System for Grant Applications • Scientific Review Group (Study Section) Initial review for scientific merit Level of support & award duration recommendations 2. Council/Institute Reviews quality of Scientific Review Recommendations to staff on funding Evaluates relevance to program priorities Advises on policy

3. Center for Scientific Review Division of Receipt and Referral • Institute assignment • Assignment to Scientific Review Group All NIH grants go here first. Important to make clear where you want the proposal to go and what Study Section to review it. Use NIH Website to familiarize with Study Sections

4. Grants Submitted to NIH Over 60,000 each year

5. Cover Letter • Suggest appropriate Institute or Multiple Institutes • Suggest appropriate Study Section www.csr.nih.gov

6. Study Sections • Standing Sections • Subject matter matches referral guidelines • 20-30 members • About 80 applications reviewed each time • Ad Hoc Special Emphasis Panels • Subject matter does not fit • Conflict of interest

7. Choosing a Study Section www.csr.nih.gov

8. Study Section Actions • Unscored (lower half) • Scored Scientific Merit Rating (priority scores and percentiles) • Deferral

9. Streamlining • Lower half based on preliminary scores • Not discussed or scored • Original written critiques provided • Not taken to Advisory Council Note: In most cases unscored applications are worth revising Persistance Persistance

10. Scored Applications • Primary, secondary, tertiary reviewers present comments • Discuss 15-20 minutes • Discuss other considerations (e.g., human subjects) • Score • Budget ?

11. Review Criteria Investigators & Resources Design & Methods Significance and Innovation Important Problem? Novel Concepts, Approach? Is Scientific Knowledge Advanced?

12. Getting the Results • Score and percentile • Summary by SRA • Receive in 4-6 weeks • Program Staff • Payline information • Advice on revision • Do not call SRA The NIH Commons

13. To Resubmit • Contact Program Officer • Funding decisions • Advice • Interpretation • Revise the Application • Organize/cluster comments • Address everything in the introduction • Don’t fight them; Be grateful and you could hit a Grand Slam!

14. NIH Grant Cycle Feb - Mar June-July Oct - Nov Receipt Dates June – July Oct – Nov Feb - Mar Review Dates Sept Feb June National Advisory Council Board Dates Dec 1 April 1 July 1 Earliest Possible Beginning Date

15. Clinical Research Applications Overall, clinical research proposals do less well than laboratory proposals. Kotchen, Lindquist, Malik, & Ehrenfeld, 2004

16. Interpretation of Review Comments Reviewers are: • Smart • Knowledgeable about science • Experienced • Overworked • Mostly sympathetic if the idea is compelling • But will likely pick 1 or 2 to champion

17. Persuasion is Key Persuasiveness requires: • Excitement • Clarity • Focus

18. Attitudes for Interpreting Comments • Assume comments helpful • Be grateful, not defensive • Be open-minded and self evaluative • If reviewers don’t get it, most likely it is your fault.

19. Preparing for Resubmission Summary Statement • You and team read and re-read • Identify questions • Organize into categories for needed revision Program Officer • Advice on resubmission • Can sometimes help with interpretation if attended review • Your Program Officer is supposed to help you Introduction Content • Organize by topic areas in logical sequence • Be sure to address all reviewer comments/recommendations Important to get several persons’ judgments on what the reviewers need to be convinced and enthused.

20. The Reviewer is Always Right! • Best to take this view even if comment seems wrong • Mostly likely wrong because there is a problem with the application • The problem? • The approach? • Clarity? • Remain a learner and use the feedback to improve the application – let reviewers help you fix it.

21. Common Criticisms • Not important or innovative • Doesn’t fit priorities • Don’t like approach and/or methods • Too ambitious • Not clear • Not justified

22. Clues for Interpretation Reviewers: Cite work not included

23. Unresponsiveness • Potentially fatal • Miss chance to improve the application • Suggests: • narrow perspective • insularity • arrogance and/or incompetence • more stubborn than interested in improving

24. Other Things to Look for in Comments • Not seeing how the proposed fits in broader picture • Questioning the scope • Questioning the complexity • Reviewer gets on a tangential issue • May have wrong Study Section

25. Reviewers Can be Wrong Appreciate the recommendation and considered the approach, however, for the following reasons we chose ………… • Usually some problem with proposal presentation • Recommendation may not be doable • Further justify and include pros and cons • Note it was considered and why not adopted

26. Complete Explanation Important • Must make rationale for everything clear • Reviewers may not be the same • May not adopt recommendations • You have limited number of chances

27. Do’s and Don’ts • Do have faith in sincerity and wisdom of reviewers. • Don’t mindlessly adopt all of their suggestions. • Do continue with preliminary work and publication while you wait for feedback. • Don’t lose focus on objectives by adding nonessential studies or measures. • Do resubmit. Very few funded on first submission.

28. You Can Change Study Sections • Not easy but can be done • Some will never fund your type of science • Ask Program Officer • May argue need certain type of expertise

29. Summary Begin early Read carefully and dispassionately Get opinions of others Have faith in the reviewers Accept comments as opportunity to improve Project enthusiasm and confidence Justify all decisions & anticipate possible questions Persist with optimism & you will SUCCEED, SMILE, & FLY HIGH