Your NIH Grant Application

1. Your NIH Grant Application 2009 MORE Program Directors Meeting June 10-12, 2009 – Colorado Springs, CO Paul Sheehy, Ph.D. Deputy Associate Director National Institute of General Medical Sciences (NIGMS)

2. ABSTRACT • The NIH has introduced major changes in processes and policies relating to grant application (submission and review) and award management. This presentation reviews the changes in grant submission (transition of various grant activities to electronic submission and migration of applications forms from PureEdge to Adobe software), grant review (new scoring system, limited resubmission, designation of New Investigator status, reformatted Critique template), and the upcoming changes to grant application forms (page limits decreased, , reformatted Critique template), and the upcoming changes to grant application forms (page limits decreased, sections aligned to review criteria, reformatted biosketch). The legal requirement to submit an electronic version of every peer-reviewed taxpayer-supported manuscript to PubMed Central (Public Access policy) is also presented. Finally, NIH and NIGMS have developed new communications vehicles to better communicate funding data and policies to the public.

3. Agenda • Enhancing Peer Review • Changes so far / ongoing / yet to come • Implications for training applications • eSubmission • Forms Changes • Upcoming transitions • Policies • What’s new • What to look for • Information Resources • What’s new • Where to look

4. Enhancing Peer Review

5. Receipt and Referral of Applications Electronic (SF424 R&R) submitted through Grants.gov Paper (PHS 398) direct to CSR to an NIH Institute CSR Referral Office assigns the application… a unique identifier (application number) to Integrated Review Group (IRG) and then a study section (SRG) Application assessed for completeness & eligibility Notice of assignment available in eRA Commons in 4 weeks. 1st Month 2nd Month

6. Review System for Grant Applications Advisory Council Assess quality of SRG process Offers recommendation to Institute Staff Evaluates program priorities and relevance Advises on policy • Scientific Review Group (SRG) • Independent outside review • Evaluate scientific merit, significance • Recommend length and level of funding 1st level Output: Priority Score and Summary Statement 2nd level 3 - 7 months Output: Funding Recommendations 1 - 3 months • Institute Director • Makes final decision based on Council input, programmatic priorities • Must also Pass Administrative Review Output: Awards or Resubmission

7. Enhancing Peer Review at NIH NIH Peer Review Virtues • Cornerstone of the NIH extramural mission • Standard of excellence worldwide • Collaboration between NIH extramural staff and scientific community Complaints • Overly burdensome • Applications too long • Process too long • Disproportionate effect of negatives • Uneven quality of reviews and reviewers

8. Enhancing Peer Review at NIH Purpose • Facilitate changing nature of science • Interdisciplinary and team science • Recognizes and utilizes new sources of information • Encourage New and Early Stage Investigators • Ease burden on research enterprise • Streamline time to award • Fund the best science, by the best scientists, with the least amount of administrative burden

9. Enhancing Peer ReviewKey Recommendations More at:http://enhancing-peer-review.nih.gov

10. Enhancing Peer Review Primary Implementation Actions • Reduce Resubmissions • Phase out second revisions • Identify and cluster New Investigator and Clinical applications • Improve Scoring • Compress the range of possible scores (10-90) • Report percentiles in whole numbers. • Improve Critiques • Shorten and focus critiques. • ALL applications receive feedback. • Enhance ReviewCriteria • Core Review Criteria – Score individual criteria. • Additional Review Criteria (scored) and Considerations (not) – reviewed as applicable • Shorten Research Plans More at: http://enhancing-peer-review.nih.gov/

11. Enhancing Peer Review at NIH Timeline Changes SO FAR Changes NOW Changes LATER January 2009 May/June 2009 January 2010 Submissions • Phase out of • A2 applications • Identification of • Early Stage • Investigator (ESI) applications • Enhanced review criteria • New scoring system • Criterion scoring • Structured critiques • Clustering Applications • Score order review • Alignment of applications & review criteria • Shorter Research Plans 11

12. Enhancing Peer Review at NIH Changes Happening So Far • Phase out of A2 applications • Identification of Early Stage Investigator (ESI) applications

13. NIH limiting all original new (i.e. never submitted) and competing renewal applications to only one resubmission. Applies to all applications submitted for January 25, 2009 due date and beyond. Previously submitted applications will be allowed two resubmissions (“grandfathered”) until January 7, 2011. Based on recommendations from the Peer Review Oversight Committee to increase the number of high quality and first resubmissions that can be funded earlier. Reduces applicant burden of multiple resubmissions. Removes delays in funding for meritorious science. More at: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-003.html Changes So Far Phase Out of A2 Applications

14. Changes So Far:Limited Resubmission of Applications Criteria for a New Application • Not a New Issue • Substantially different in content and scope • Fundamental changes in questions and/or outcomes • Rewording of the Title and Specific Aims or incorporating minor changes in response to reviewer comments does not constitute substantial change • Request for a different review committee or a different IC are not sufficient • Important implications for training , policies have not yet been worked out • Applications changing activity codes are new submissions • Change in activity code • Applications to RFAs (usually) including ARRA

15. Expectations for preliminary data or track record should not be the same as for established investigators Encourage earlier transition to research independence Strongly encourage New Investigators, particularly ESIs, to apply for R01 grants when seeking first-time NIH funding  Clustering facilitates consistency Changes So Far:New/Early Stage Investigators Purpose

16. Changes So Far:New/Early Stage Investigators Definitions • New Investigator (NI) is a PD/PI who has not yet competed successfully for a substantial NIH research grant • Receipt of the following awards does not remove new investigator status: R00, R03, R15, R21, R34, R36, R41, R42, R55, R56, SC2, SC3, all Fs, most Ks, all Loan Repayment contracts, G07, G08, G11, G13, G20, S10, S15, S21, S22. • Early Stage Investigator is NI who is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent) • Both eligible for the Shortened Review Cycle option • Status applies only to R01s • For multiple PD/PIs: all PD/PIs must meet definition

17. Enhancing Peer Review at NIH Changes Happening NOW • Enhanced review criteria • New scoring system • Criterion scoring • Structured critiques • Clustering of New Investigator applications • Score order of review • Implemented for applications submitted: • for Fiscal Year 2010 funding consideration • for Recovery Act (ARRA) RFAs

18. Changes Happening NOW:Enhanced Review Criteria Overall Impact/PriorityScore Reflects the reviewers’ assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved • In consideration of: • Core criteria • Additional review criteria (RFA or PAR) • Additional review criteria (scored) – as applicable • Additional review considerations (not scored)

19. Changes Happening NOW:Enhanced Review Criteria Overall Impact  Overall Impact/Priority Score • Core criteria order: • Significance* • Investigator(s)* • Innovation* • Approach* • Environment* • Additional review criteria • Protections for Human Subjects • Inclusion of Women, Minorities, and Children • Vertebrate Animals • Resubmission Applications • Renewal Applications • Biohazards *Will receive individual criterion scores Side-by-side comparison http://grants.nih.gov/grants/peer_review_process.htm

20. Changes Happening NOW:Enhanced Review Criteria Significance • Does the project address an important problem or a critical barrier to progress in the field? • If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? • How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

21. Changes Happening NOW:Enhanced Review Criteria Investigator • Are the PD/PIs, collaborators, and other researchers well suited to the project? • If Early Stage Investigators or New Investigators, do they have appropriate experience and training? • If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? • If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project*? (*Moved from Approach)

22. Changes Happening NOW:Enhanced Review Criteria Innovation • Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? • Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? • Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?

23. Changes Happening NOW:Enhanced Review Criteria Approach • Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? • Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? • Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed? • Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? • Are potential problems, alternative strategies, and benchmarks for success presented? • If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed? • If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed?

24. Changes Happening NOW:Enhanced Review Criteria Environment • Will the scientific environment in which the work will be done contribute to the probability of success? • Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? • Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?

25. Changes Happening NOW:Enhanced Review Criteria Additional Review Considerations • As applicable for the project proposed • Reviewers will address each item • Reviewers will not give scores for these items should not consider them in providing an overall impact score. • Budget and Period Support • Select Agent Research • Applications from Foreign Organizations • Resource Sharing Plans

26. Changes Happening NOW: New Scoring System 9-Point Scale • Reduces number of rating discriminations • Provides rating descriptors • To improve reliability • To encourage use of the entire range • Will be used for: • Overall impact/priority scores • Individual criterion scores • Will be implemented for applications submitted for: • FY2010 funding consideration and beyond • Recovery Act funding opportunity announcements

27. Changes Happening NOW: New Scoring System New Score Descriptors

28. Enhancing Peer Review at NIH Structured Critiques • Decrease variability • Increase quality of information in critiques • More succinct, better organized • Encourage evaluative statements • Ensure that reviewers address all review criteria and considerations

29. Phases of Process Changes Happening NOW: Structured Critiques Critique Templates • Bulleted comments • Scores for five review criteria • Required comments: • Protections for Human Subjects • Inclusion Plans • Vertebrate Animal Welfare • Biohazards • Budget Links to definitions of review criteria

30. Changes Happening NOW:Score Order Review Order of Discussion Where feasible, discussion order based on: • Clustering of New Investigator applications • Clustering of clinical applications • Clustering of similar activity codes • Preliminary overall impact/priority scores

31. Changes Happening NOW Additional Information • Enhancing Peer Review Website: • (http://enhancing-peer-review.nih.gov/index.html) • NOT-OD-09-025: • http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-025.html • Side-by-side comparison of enhanced and former review criteria • http://grants.nih.gov/grants/peer_review_process.htm • Guidelines for Reviewers • http://grants.nih.gov/grants/peer/reviewer_guidelines.htm

32. Happening LATER: Application Changes Beginning with Jan 2010 Submissions • Alignment of applications & review criteria • Shorter Research/Research Training Plans *NIH Plans a Fall Release of new Forms

33. Happening LATER:Application Changes Alignment of Applications and Review Criteria Changes to three parts of application: • Biographical sketch • Research/Research Training Plan • Resources

34. Happening LATER:Application Changes Biographical Sketch • Personal Statement – why experience and qualifications make the applicant particularly well-suited for role in the project • Publications limited to 15 • 5 most recent • 5 best • 5 most relevant to the application

35. Happening LATER:Application Changes Restructured Research Plan Introduction (amended appls only) Specific Aims Background and Significance Preliminary Studies/Progress Report Research/Research Training Design and Methods Inclusion Enrollment Report (as applicable) Progress Report Publication List Human Subjects Sections…. protections, women/minorities, enrollment, children Other Research Plan Sections…. animals, select agent, MPI, consortium, support, sharing Appendix Research Strategy

36. Happening LATER:Application Changes Current Application New Application

37. Happening LATER:Application Changes Facilities and Other Resources (in 424 part of the R&R Other Project Information; in 398 the Resources Format Page) Environment - New instruction to address how scientific/training environment will contribute to probability of success, unique features of environment, etc. For ESIs, provide description of institutional investment in success of the investigator.

38. Happening LATER: Alignment of Application with Review Criteria Core Review Criteria Application

39. Happening LATER:Shorter Applications Page Limit Revisions

40. Adobe Application Forms and Electronic Submission Changes to electronic applications are on the way. Learn how this affects YOU today!

41. Adobe: Not very different… • The new Adobe forms look very similar to PureEdge forms -- Changes are cosmetic and navigational (see comparison at: http://era.nih.gov/ElectronicReceipt/files/PureEdge_v_Adobe.pdf • Overall electronic submission process remains the same: • Find opportunity • Download application package • Develop research plan and other PDF attachments • Complete forms • Submit application • Check assembled application in eRA Commons Works on both Macs and PCs

42. … but follow these important tips to ensure success • Adobe Reader 8.1.3 or 9.0 requiredtoopen forms • Stay tuned to Grants.gov’sDownload Software page for changes (http://www.grants.gov/help/download_software.jsp) • A pop-up usually warns if you have a wrong version • If using an Adobe Acrobat product to create PDFs, check Grants.gov’s Web site for help on settings • To ensure the application reader opens in the correct version of Adobe

43. Reminder: Avoid Common eSubmission Errors • Use PDF format for text attachments and do not embed movies or other materials in PDF attachments. • R&R Senior/Key Person Profile(s) form • Include eRA Commons Username in the ‘Credential, e.g., agency login’ field for all individuals assigned a PD/PI role. • Include the Organization Name for all Senior/Key Persons listed. • R&R Budget form - Senior/Key Person effort must be greater than zero. • http://era.nih.gov/ElectronicReceipt/app_help.htm Clicking ‘Submit’ is not the last step. Remember. . . if you can’t view it (in Commons), we can’t review it ! More at: http://era.nih.gov/ElectronicReceipt/avoiding_errors.htm

44. Electronic Submission: Update • Adobe-based grant application forms are now available for all FOAs requiring eSubmission. • Career Development Awards (Ks) transitioned to electronic submission. • Next set of transitions: • New PHS 398 and SF 424 (R&R) Application forms expected in Fall, 2009 • Fellowship (F): August 8, 2009 (forms soon to be released ) • Training (T & K12): January 25, 2010 (Tentative)

45. Support Resources & Help Desks • Contact Grants.gov Contact Center for questions on form functionality or submission of the forms to Grants.gov.  • http://grants.gov/applicants/app_help_reso.jsp • Contact the eRA Help Desk at NIH for technical issues that threaten NIH’s timely receipt of your application. • Work with the Grants.gov Contact Center and be sure to document the issue and provide NIH with the tracking number received from Grants.gov Contact Center. • http://era.nih.gov/ElectronicReceipt/ • http://era.nih.gov/ElectronicReceipt/app_help.htm

46. New Scientific and Grants Management Policies

47. Compliance mandated by Law Applicable to peer-reviewed articles accepted for publication on or after 4/7/08 Submission of articles to NIH: Deposit final manuscript in NIH Manuscript Submission (NIHMS) system. Many journals will submit articles on behalf of author. Articles must be publicly available on PubMed Central no later than 12 months after date of publication Citing Articles in Applications & Progress Reports: Beginning with 5/25/08 submission date, use the PubMed Central ID or NIH MS ID number for each article. More at: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-071.html Public Access Policy “All investigators funded by NIH must submit to PubMed Central an electronic version of their final, peer-reviewed manuscripts upon acceptance for publication.”

48. Helpful NIH Technical Assistance Resources

49. New reports, data and analyses website released in March, 2008 and expanded significantly in January, 2009 with addition of RCDC data (Research, Condition and Disease Categorization process) Replaces the current Award Information and Data web pages (including CRISP) and will provide: Quick access to “Frequently Requested Reports” FAQs on how success rates are computed and questions on the NIH budget Search tools for locating data and reports quickly and easily Links to funding estimates for certain research areas, conditions, and diseases. Foundation for broader NIH-wide Research Portfolio Online Reporting Tool (RePORT) http://report.nih.gov/ More at:http://report.nih.gov New Tool to Search NIH Funding

50. Weekly Updates from NIGMS Funding Opportunities Meetings Resources Other useful information Interactive Community discussion is encouraged through comments Sample Content Early Notice: New Microbial-Host Interactions Grants (Upcoming RFA) Discussion of the Scientific Workforce FY 2010 NIGMS Budget Request FAQs Available via email, web and RSS More at:http://loop.nigms.nih.gov NIGMS Feedback Loop