“Anatomy” of Grant Process

1. NIH Peer Review &Grant Writing for SuccessMichael A. Sesma, Ph.D.National Institute of Mental Health Roger G. Sorensen, Ph.D., M.P.A.National Institute on Drug Abusewith inspiration from Coelho, Perl, Frascella & Levitin2008 NIH Mini-Regional, Albany

2. CSR Referral and Review “Anatomy” of Grant Process Researcher Idea Institution Program Announcement or RFA Program Staff Collaborators Grant Application (R01, R03, R21, K01, K08, etc.) Revision $ National Advisory Council Program Staff

3. The NIH Peer Review Process Division of Receipt and Referral (DRR), Center for Scientific Review (CSR) Application received by CSR DRR Assignments made   Initial peer review Funding considerations SRG (CSR or IC) IC(s) – duals possible  Second level of review National Advisory Council/Board  IC Director (funding decisions)

4. The NIH Peer Review Process Division of Receipt and Referral, CSR • Checks for completeness • Determines area of research • Assigns an identification number • Assigns a grant number • Assigns application to specific • NIH IC for possible funding • Assigns a Scientific Review Group NIH receives ~ 80,000 applications per year Central receiving point for all competing applications

5. The NIH Peer Review Process CSR Review Most R01s, fellowships, and small business applications Some Program Announcements (PAs, PARs), Requests for Applications (RFAs) Institute/Center Review IC-specific features Program projects Training grants Career development awards RFAs Receipt and Referral

6. The NIH Peer Review Process First level of review Scientific Review Group (SRG) - Provides initial scientific merit review of grant applications - Rates applications and makes recommendations concerning level of support and duration of award Second level of review Advisory Council - Makes recommendations to IC staff concerning funding - Evaluates program priorities and relevance - Advises on policy

7. The NIH Peer Review Process An Extramural Scientist responsible to NIH for the scientific and technical review of applications Designated Federal Official manages SRG, ensures fair and unbiased evaluation of scientific and technical merit of applications for grants, contracts, cooperative agreements Identifies and assigns reviewers to the SRG Provide accurate summaries of the evaluation to aid funding recommendations made by National Advisory Councils to Institute Directors Point of contact for applicants during the review process (i.e., after submission, until review is completed) Scientific Review Officer (SRO)

8. The NIH Peer Review Process Scientific Review Groups (SRGs) • Make recommendations: • Scientific and technical merit • Budget and project duration • Bars to award – human subjects, • vertebrate animals, biohazards • Resource Sharing Plans • Other administrative factors • Priority scores or UN • Written critiques (summary statements) Study Sections do not make funding decisions!

9. Who are the Peer Reviewers? Demonstrated Scientific Expertise Doctoral Degree or Equivalent Mature Judgment Work Effectively in a Group Context Breadth of Perspective Impartiality Interest in Serving Adequate Representation of Women and Minority Scientists

10. The NIH Peer Review Process Reviewer Assignments • ≥ Three qualified reviewers (2 + 1) • Based on scientific content of • application • Expertise of reviewer • Suggestions from PI on types of • expertise – not names! • Suggestions from Program staff • Conflicts of interest Telephone reviewers – discuss and score Mail-in reviewers – do not discuss, do not score

11. The NIH Peer Review Process Remember Review Criteria • Included in the Funding • Opportunity Announcement (FOA) • Sent to reviewers with applications • Guide discussion at the meeting • Format for critiques in summary statement • Standardized across NIH by type of • mechanism

12. The NIH Peer Review Process Remember Review Criteria • Standard criteria*:Factored into priority score • Significance * Research projects and contracts • Approach • Innovation • Investigator(s) • Environment • Protection of human subjects • from research risks • Vertebrate animal welfare • Biohazards

13. The NIH Peer Review Process • Significance:Does the study address an important problem? How will scientific knowledge be advanced? • Approach:Are design and methods well-developed and appropriate? Are problem areas addressed? • Innovation:Are there novel concepts or approaches? Are the aims original and innovative? • Investigator:Is the investigator appropriately trained? • Environment:Does the scientific environment contribute to the probability of success? Are there unique features of the scientific environment?

14. The NIH Peer Review Process Review Criteria • Additional considerations: • Not factored into priority score • Budget • Foreign institutions • Resource Sharing Establishment of Multiple Principal Investigator Awards for the Support of Team Science Projects (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-07-017.html) Encouraging Early Transition to Research Independence: Modifying the NIH New Investigator Policy to Identify Early Stage Investigators (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-08-121.html)

15. The NIH Peer Review Process Research Plans • Introduction to Application • (resubmission or revision only) • Specific Aims • Background and Significance • Preliminary Studies/Progress Report • Research Design and Methods • Inclusion Enrollment Report • (renewal or revision only) • Bibliography and References Cited

16. The NIH Peer Review Process Research Plans - continued • Protection of Human Subjects • Inclusion of Women and Minorities • Targeted/Planned Enrollment Table • Inclusion of Children • Vertebrate Animals • Select Agent Research • Multiple PD/PI Leadership Plan • Consortium/Contractual Arrangements • Letters of Support • Resource Sharing Plans

17. The NIH Peer Review Process SRG Meeting Procedures • Call to Order - Chairperson • Policy and instructions – SRO • Conflicts of Interest • Confidentiality • Streamlining • Discuss each application • Scoring • Discuss other considerations • Resource Sharing Plans • Foreign institutions

18. The NIH Peer Review Process SRG Meeting Procedures • Discussion format • Members with conflicts excused • Initial levels of enthusiasm (assigned reviewers) • Primary reviewer explains project, strengths/weaknesses • Other assigned reviewers follow • Open discussion (full panel) • Levels of enthusiasm (assigned reviewers) • Reviewer workload • ~ 6 – 8 as “reviewer” • ~ 2 – 3 as “discussant”

19. The NIH Peer Review Process SRG Meeting Procedures • If 60 applications/SRG meeting • ~ 50% streamlined • 30 applications to discuss and score • If 9 hour SRG meeting (8:00 AM – 5:00 PM) • ~ ½ hour introduction, streamlining • ~ 1 hour lunch, 2 x 15 minute breaks Review Implications ~ 14 minutes/application ~ 3 - 4 minutes/reviewer Clarity and brevity are essential!

20. SRG/Study Section Actions Scored, Scientific Merit Rating (priority scores and percentiles) Unscored (lower half) Deferral The NIH Peer Review Process

21. The NIH Peer Review Process eRA Commons Post Review • Priority Score • Three days after conclusion of SRG meeting • Summary statement • 4 – 8 weeks after conclusion of SRG meeting • Available to Program Officers at that time • Confidential document • Available to • PD/PIs • NIH officials • Advisory Council members

22. The NIH Peer Review Process Summary Statement • Program Officer, Contact information • Priority Score, Percentile (if applicable) • Description (provided by applicant) • Resumé and Summary of Discussion • Reviewer critiques – essentially unedited • Committee Recommendations • Budget • Human subjects • Vertebrate animals • Inclusion plans • Administrative Notes

23. The NIH Peer Review Process After the Review • Program Officer = Point of Contact • Wait for summary statement • Read summary statement carefully, several times, take notes • ………then contact program officer The score/percentile rank is not a guarantee of funding!

24. The NIH Peer Review Process After the Review • Consult Program Officer • Consider options if outcome • unfavorable • Revise and resubmit application • Consider critiques in summary statement • Address critiques in introduction and text • Appeal review outcome • Procedural deficiencies • Factual errors • May result in re-review of same application by different SRG

25. The NIH Peer Review Process After the Review • Consult Program Officer • If an award is likely the program officer will • Discuss responses to issues raised by reviewers • Budget • Human subjects or animal welfare issues • Inclusion issues • Administrative requirements • Negotiate Funding Amounts • Study Section Recommendations • NIH and Institute Guidance

26. The NIH Peer Review Process Advisory Council/Board • Second level of review • Advisory to IC Director • Rosters:http://www1.od.nih.gov/cmo/committee/index.html • Schedule:http://www1.od.nih.gov/cmo/committee/index.html

27. The NIH Peer Review Process Advisory Council/Board • Panel of scientific experts and lay members provide advice to Institute Director on • Research priority areas • Funding • Policy and program development • Appeals of review recommendations • Quality of SRG review • Concur with SRG recommendations • Modify SRG recommendations • Deferral for re-review • Cannot change priority score from SRG

28. The NIH Peer Review Process Funding Considerations • Scientific and technical merit • (initial peer review) • Council recommendation • Relevance to IC program priorities • Availability of funds

29. Who Actually Makes the Funding Decisions? The Institute Director! Factors Considered: Scientific Merit Contribution to Institute Mission Advisory Council Recommendation Program Balance Availability of Funds

30. The NIH Peer Review Process Additional Information • Enhancing Peer Review Initiative • http://enhancing-peer-review.nih.gov/ • Office of Extramural Research • Peer Review Process • http://grants.nih.gov/grants/peer_review_process.htm • Peer Review Policies & Practices • http://grants.nih.gov/grants/peer/peer.htm • Center for Scientific Review • http://cms.csr.nih.gov/AboutCSR/Welcome+to+CSR/

31. Changes are coming to Peer Review http://enhancing-peer-review.nih.gov/

32. The NIH Peer Review Process • Enhancing Peer Review at NIH • The preliminary implementation plans for 2009-2010 calendar years are: • Engage the Best Reviewers • Recruit and retain the best reviewers; enhance reviewer training • Increase flexibility for reviewers through virtual reviews • Improve the Quality and Transparency of Review • Improve scoring transparency and scale • Score streamlined applications • Shorten and restructure applications – e.g., 12 page R01 research plan • Ensure balanced and fair reviews across scientific fields and career stages, and reduce administrative burden • Fund the best science earlier, reduce resubmissions • Review like applications together, e.g., ESI or clinically focused projects http://www.nih.gov/news/health/sep2008/od-30a.htm

33. Grant Writing for Success Writing the Application • Start early • Seek advice from colleagues • Start with a good idea • Talk to your NIH Program Official(s) • Use the NIH webpage (www.nih.gov) • Remember review criteria • Follow instructions carefully Transition to Electronic Submission (http://era.nih.gov/ElectronicReceipt/)

34. Urban Myth of Grantsmanship It is not a process by which bad ideas get transformed into good ones … Grant Writing for Success … rather, it is more often the case of a good idea disguised as a bad one.

35. Principles of Success Understand the peer review process Understand the agency mission Every IC is different! Secure collaborators (mentors) to complement your expertise and experience Don’t compete … collaborate! Learn and practice the skills of writing applications for grant funds

36. Remember … Before you start Talk to Program Staff at appropriate IC Read instructions for application form SF 424 R & R or PHS 398 Know your audience Which review committee is most likely to get your application? Propose research about which you are passionate and totally committed to doing

37. The Formula for Writing a Successful Grant Application

38. Good Idea Does it address an important problem? Will scientific knowledge be advanced? Does it build upon or expand current knowledge? Is it feasible … to implement? to investigate?

39. Good Grantsmanship • Grant writing is a learned skill • Writing grant applications, standard operating protocols and manuals of procedures that get approved are learned skills • Writing manuscripts that get published in peer reviewed journals is a learned skill • Grantsmanship is a full time job • Learn about the grant application process

40. Good Grantsmanship Contact NIH program staff early Assess IC interest & “goodness of fit” Are there related FOAs? Searching NIH web sites is good start … but follow up with personal contact

41. Good Grantsmanship • Show your draft application to a colleague • Show your draft application to a colleague who does not already know what you intend to do • Show your draft application to a colleague who is not your best friend

42. Good Grantsmanship • Your draft reviewers need to understand • What you intend to do • Why you believe it is important to do • Exactly how you are going to do it • If they don’t get it, you must revise your application • Leave enough time to make revisions

43. Good Presentation • Read the application instructions carefully • Read the application instructions carefully • Don’t forget … ... read the application instructions carefully 3 Simple Steps:

44. Good Presentation Organize the Research Plan to answer 4 essential questions: What do you intend to do? Why is the work important? What has already been done? How are you going to do the work?

45. Good Presentation Address the 5 review criteria Significance Approach Innovation Investigator Environment

46. Good Presentation Provide well-focused research plan Keep specific aims simple … and specific Link hypotheses to specific aims Explain method to test every hypothesis Don’t wander from the main theme A conceptual model can clarify ideas

47. Good Presentation Be realistic … not overly ambitious Discuss potential problem areas Discuss possible solutions Explain rationale for your decisions Be explicit Reviewers cannot read your mind … Don’t assume they know what you intend

48. Good Review Get to the right review group Title, abstract, specific aims all point to the main goals of your project Attach a cover letter suggest IC and review group assignment* outline areas of key expertise needed for appropriate review do not name specific reviewers * Consult with Program Officer

49. Good Luck Results from: Good Ideas Good Grantsmanship Good Presentation Good Review