Just When You Have it Down, NIH Changes the Format!!!

1. Just When You Have it Down, NIH Changes the Format!!! HSC Research Forum 4 p.m. December 15, 2009 Bev Daly and Russ Prough

2. The diagnostic phase involved an in-depth evaluation of the current NIH peer review system.  In June 2007, the NIH Director established two working groups:   External – The Advisory Committee to the Director Working Group (ACD WG), co-chaired by Dr. Keith Yamamoto of the University of California, San Francisco, and Dr. Lawrence Tabak, Director of the NIH National Institute of Dental and Craniofacial Research (NIDCR); and Internal – The Steering Committee Working Group (SC WG), co-chaired by Dr. Tabak and Dr. Jeremy Berg, Director of the NIH National Institute of General Medical Sciences (NIGMS).  The working groups solicited formal input from key stakeholders and deliberated on challenges and recommendations. The final report (PDF - 1.61 MB) of the NIH 2007-2008 Peer Review Self-Study was submitted to the NIH Director on February 29, 2008. The study identified seven major challenges and described recommendations to address these challenges: Challenge 1: Reducing Administrative Burden on Applicants, Reviewers, and NIH Staff Challenge 2: Enhancing the Rating System Challenge 3: Enhancing Review and Reviewer Quality Challenge 4: Optimizing Support for Different Career Stages and Types Challenge 5: Optimizing Support for Different Types and Approaches of Science Challenge 6: Reducing the Stress on the Support System of Science Challenge 7: Meeting the Need for Continuous Review of Peer Review Diagnostic Phase

3. Design Implementation Phase In March 2008, the NIH Director established the Steering Committee Peer Review Implementation Group to draft implementation plans for each recommended action. Subgroups of the Steering Committee, which consisted of Institute Directors, NIH program and review officers, planning and evaluation experts and statisticians, solicited feedback from both NIH internal and external communities. This feedback, together with careful consideration of the pros and cons of both individual and combined recommendations, informed decisions on enhancements to the peer review system.    On June 6th, 2008, the Peer Review Enhancements and Implementation Plan was announced in a Press Release, and at the Advisory Committee to the Director (ACD) meeting The Implementation Plan is organized into the following priority areas: Priority 1 – Engage the Best Reviewers - The excellence of peer review is directly correlated with the ability to recruit and retain the most accomplished, broad-thinking, and creative scientists to serve on NIH study sections. Priority 2 – Quality & Transparency of Review - The peer review process must strive for maximum clarity, fairness, and consistency and help applicants determine a best course of action once reviewed. The process of review should focus on the potential impact, originality, and feasibility of the proposed research. Priority 3 – Provide Balanced and Fair Reviews Across Scientific Fields and Career Stages - Peer review should fairly evaluate proposals from all scientists, regardless of their career stage or discipline, and avoid bias towards more conservative and proven approaches at the expense of innovation and originality. Priority 4 – Continuous Review of Peer Review - The last priority is to develop a permanent process for continuous review of peer review. Peer review should continuously adapt itself to the evolution of science. The NIH peer review process will commit to a continuous quality control and improvement process based on a rigorous and independent prospective evaluation that favors innovative approaches to review and program management.

4. NIH Resources http://enhancing-peer-review.nih.gov • Video – Overview of Enhancements • Frequently Asked Questions • Details of Application Changes

5. Enhancing Peer Review Overview and Timeline • Phase out of A2 applications • Identification of Early Stage Investigator (ESI) R01 applications • Enhanced review criteria • New scoring system • Criterion scoring • Structured critiques • Score order review • Clustering of New Inv. Applications • Restructured Applications • Shorter Page Limits and New Instructions 5

6. Major Changes to ApplicationsFor due dates on or after Jan 25, 2010 • Restructured Application Forms • Shorter Page Limits and New Instructions For ALL competing applications: New, Renewal, Revision, and Resubmission

7. Goals of Shortened Page Limits • Reduce burden • Focus on the essentials of the science • Avoid information overload

8. Overview of Shorter Page Limits Note: Follow FOA page limit requirements if different from the application instructions. Full table of page limits available at: http://enhancing-peer-review.nih.gov/page_limits.html

9. Restructured Application Forms • Introductions limited to one page • Specific Aims limited to one page • Research Strategy

10. Major Changes to the Research Plan • Specific Aims will include new language about the impact of the proposed research. • Research Strategy will be created as a new section and will include 3 of the current sections • Background and Significance • Preliminary Studies/Progress Report • Research Design and Methods

11. New Research Plan Components Introduction Specific Aims Background and Significance Preliminary Studies/Progress Report Research Design and Methods Inclusion Enrollment Report Bibliography and References Cited Human Subjects Sections…. protections, women/minorities, enrollment, children Other Research Plan Sections…. animals, select agents, multi PD/PI, consortium, support, resource sharing Appendix Research Strategy

12. Research Strategy Significance: • Explain the importance of the problem • Explain how the project will improve scientific knowledge, technical capability and/or clinical practice • Describe how the concepts, methods, interventions, etc. that drive the scientific field will be changed if the aims are achieved.

13. Research Strategy Innovation: • Explain how the proposed work will shift current research or clinical practice • Describe any novel concepts, approaches, instrumentation or intervention(s) and any advantage over existing

14. Research Strategy Approach: • Describe the overall strategy, methodology, and analyses to accomplish aims • Discuss potential problems, alternative strategies, and benchmarks for success • Explain strategy for high risk aspects of proposed work

15. Research Strategies Approach: • Include Preliminary Studies in Approach Section • Describe any issues related to hazardous materials and precautions to be made • Include a Progress Report for Renewal and Revised Applications

16. Changes to Biographical Sketch • Personal Statement added: • “Briefly describe why your experience and qualifications make you particularly well-suited for your role in the project” • Publications revised: • Limit the list of publications or manuscripts to no more than 15 • Applicant is encouraged to make selections based on recency, importance to the field, and/or relevance to the application

17. Changes to Resources and Facilities • Instructions added to Resources: • Provide a description of how the scientific environment will contribute to the probability of success of the project • For Early Stage Investigators (ESIs), describe the institutional investment in the success of the investigator

18. Steps for Success – Part 1 • Read about the upcoming requirement changes now so that you can begin writing your Research Strategy • Information available on the Enhancing Peer Review website includes: • Policy Announcements: NOT-OD-09-149 NOT-OD-10-002 • Details of Application Changes • Training & Communications Resources http://enhancing-peer-review.nih.gov/

19. Steps for Success – Part 2 • In December, go back to the updated FOA or reissued Parent Announcement • For both electronic and paper, choose the correct application package and instructions to download: • SF 424 (R&R): ADOBE_FORMS_B • PHS 398: Revision date “June 2009” Applications submitted using incorrect forms will be delayed and may not be reviewed!

20. Steps for Success – Part 3 • Read the new application instructions carefully • For due dates on or after January 25, 2010, submit your electronic and paper applications using the new application forms

21. What Has Not Changed • Need to have a good idea about how to answer an important question • Reviewers need to be able to understand WHAT you want to do, WHY it is important, and can YOU do it? • Need to align YOUR goals with the funding agency goals.

22. Five Broad Themes Will Define NIH's Future, F. Collins, M.D. Director of NIH • Applying unprecedented opportunities in genomics and other high throughput technologies to understand fundamental biology and to uncover the causes of specific diseases. • Translating basic science discoveries into new and better treatments and practice. • Putting science to work for the benefit of healthcare reform. • Encouraging a greater focus on global health. • Reinvigorating and empowering the biomedical research community.

23. New NIH Biosketch Requirements A. Personal Statement: Describe your training and expertise, how it related to the proposed project B. Positions and Honors (same as before) • Selected Peer-reviewed Publications (summarize as selected from X peer-reviewed publications: No more than 15 with Pubmed designations) 1. Most relevant to the current application 2. Additional recent publications of importance to the field (in chronological order) D. Research Support 1. Ongoing Research Support (current and future) 2. Completed research Support (last 5 years) E. Four pages maximum

24. New requirements for RCR Training • Manditory training for all Trainees • IPIBS Predoctoral Fellows: BIOC 630-01: Research Ethics-Spring Semester (required of IPIBS fellows) • Post-doctoral Fellows: Research!Louisville Kick-off of RCR training (required) • A paragraph covering this will be provided in late December for P.I.’s to include in their grants

25. Research Compliance Aspects • Requests specific information about: • IACUC related issues • IRB related issues • Biohazard related issues • Sharing • Statistical Analysis

26. NIH Review System • Grants submitted electronically using 424 forms: Portal 5 p.m./5 day OGM requirement • CSR Regular Study Sections and Acronyms http://www.csr.nih.gov/Roster_proto/sectionI.asp • Chartered vs. non-chartered • Fellowship Review Committees • SBIR/STIR • Reviewers will do all reviews using the ERA Commons Accounts • Currently seeking qualified reviewers

27. Scoring System and Procedure • 9-point rating for impact and priority with 1 = Exceptional and 9 = poor (whole numbers) • Review Criterion: • Significance • Investigators • Innovation • Approach • Environment • Overall Impact/Priority Score

28. Strengths & Weakness Scoring Table

29. Interpreting your Scores • Too early to tell what cutoffs will be • The 1-9 score will be multiplied by 10 for your final impact/priority score (10-30 Exceptional to Excellent) • How low in priority the score will go depends on NIH budget increase • Other Translational Program Goals

30. Some considerations • Will favor experienced writer • Logic of explaining innovation and significance had to be emphasized • With shortened review, past performance will be importance and you will have to tell them specifically about past performance in Biosketch • Approach statement critical

31. Final Comments • A question and answer session for those submitting for February 5, 2010 • Challenge GrantsRO1 • Grants Management, Bev Daly, Rita Wedig, Russ Prough, Peter Rowell, Others