The NIH review process

1. The NIH review process Kathryn Lee, RN, PhD April 3, 2009 MDP

3. Grant Mechanisms R series (research projects) • R01 -individual research • R03- small, short-term • R21- exploratory/developmental • R15 – certain institutes with little NIH funding

4. 1946 The Last NIH Study Section 2008 One Review Platform for 62 years The First NIH Study Section

5. The Letter of Intent (LOI) Used by federal agencies/foundations • Filter applications to their interest area • Appoint appropriate reviewers Specific to the agency, typically ask for • Abstract • NIH Biosketch • Nomination letter (some, but not all)

6. Writing an LOI- Start Early • Limited to 1-2 pages • Title of proposal • Background of applicant (or Biosketch) • Objectives • Design and Methods • Statistical analysis plan

7. LOI: Can really help your process • finalize key aims/questions • get prepared to submit a well designed application in short time frame • Start a relationship with your future program officer

8. Letters of Intent They’re Not Easy ! “If I had more time, I would have written you a shorter letter.” Mark Twain

9. National Institutes of Health • NIH Mission: • Promote biomedical and behavioral research to help improve the health of all Americans • Carried out through 27 Institutes and Centers

10. FY 2007 NIH Budget is $28.6 Billion Spending at NIH $4.5 B Spending Outside NIH $24.1 B

11. Goal of Peer Review Independent, fair, thorough, and competent review of each application Identify and rank appropriately those applications that show the greatest promise of advancing biomedical science and/or improving disease prevention, diagnosis and treatment

12. Role of Reviewer • Provide judgment of the scientific merit of each application; NOT TO DETERMINE FUNDING • Criteria for selecting reviewers • Record of scientific excellence • Able to see “big picture” • Fair and balanced • Willing to follow guidelines and stay w/in required time frame • Articulate opinions in a clear, concise manner • Open minded to the views of other reviewers

13. Conflict of Interest • Worked with key personnel in past 3 years or currently • Financial gain • Close relative of key personnel • PI is from your institution • Recognized scientific disagreement • PI was your student/major dissertation advisor • You are applying for job at PI’s institution • PI is applying for job at your institution

14. Review Details • Each application 3-4 assigned reviewers • Primary , Secondary – full written critique • Discussant(s)- summary paragraph

15. Tips for Reviewers • Focus on science, not “grantsmanship” • Keep in mind “big picture” • Distinguish between major problems and minor concerns or differences in approach • “lack of detail” – what is missing and why it matters • “Trust me” proposal

16. Written Critiques • Address each of 5 review criteria • Address human subjects/inclusion • Overall evaluation/summary paragraph • Be specific, constructive, and concise • Not a time for mentoring • Do not identify yourself

17. Review Criteria • Significance 1= exceptional 9 = poor • Approach (1 - 9) • Innovation (1 - 9) • Investigators (1 - 9) • Environment (1 - 9) • Overall IMPACT

18. New Scoring Overall Impact Score Guidance on weighing strengths and weaknesses High Impact 1 Exceptional Strengths 2 Outstanding 3 Excellent Moderate Impact 4 Very Good 5 Good 6 Satisfactory Low Impact 7 Fair 8 Marginal Weaknesses Poor 9

19. Chair’s Role • Ensure that all appropriate viewpoints are expressed • Ensure that discussion is fair, balanced, and appropriate • Promote consistent scoring • Summarize panel’s views • Time management

20. Percentiles • Scores are translated into percentiles which are used to make funding decisions • Currently most institutes funding at 8-15th percentile • Score clustering results makes it easier to fund out of order

21. Dual Review System for Applications 2nd Level of Review NIH Institute/Center Council • 1st Level of Review • Scientific Review Group (SRG)

22. Streamlining • Purpose: make more efficient use of time at meetings • Goal: streamline 50-60%, so proposals that are not competitive are not discussed • Streamlining does NOT equal BAD

23. CSR Peer Review: 2008 77,000 applications received 56,000 applications reviewed 16,000 reviewers 240 Scientific Review Officers 1,600 review meetings

24. Major Complaints About NIH Peer Review Process is too slow Not enough senior/experienced reviewers Process favors predictable research instead of significant, innovative, or transformative research Time and effort required to write and review are a heavy burden on applicants and reviewers

25. Assign Applications Accurately & Efficiently Retooled for electronic submission Applications are now submitted electronically Assign applications using text fingerprinting, and text mining programs Full Implementation by early 2009

26. Fund best research earlier & reduce burden on applicants, reviewers, & NIH More flexible deadlines Abolish A2 applications

27. Improve Quality & Transparency of Peer Review May-July meetings 2009 Shorten summary statements, follow template for each criteria Change the rating system Use 1-9 integers Score each criteria Provide score for all applications (even those not discussed) Spring 2010 Shorten applications, aligning with review criteria Impact, investigator, innovation (if applicable), research strategy, facilities

28. Number of Applications Submitted Historical Growth

29. Reviewer’s Load Applications Per Reviewer October Council Rounds

30. RO1 Resubmission Within 4 Months of Original Application

31. Confidentiality • “What happens in study section stays in study section” • Materials are proprietary • Don’t discuss outside of the meeting • Don’t show application to anyone else • Avoid web sites associated with grant

32. NIH resources • Proposal writing guides • Avoiding common mistakes in an application • SON website link to NIH website at:  http://nurseweb.ucsf.edu/www/ix-rs.shtml.