150 likes | 272 Views
First Mondays Career Development. Center for Education and Career Development (CECD) NUCATS Institute Clinical and Translational Sciences (CTSA) Faculty Affairs Office, FSM. Adapting to the New NIH Review and Submission Policies December 7, 2009. Rick McGee, PhD Lew Smith, MD
E N D
First MondaysCareer Development Center for Education and Career Development (CECD) NUCATS Institute Clinical and Translational Sciences (CTSA) Faculty Affairs Office, FSM
Adapting to the New NIH Review and Submission PoliciesDecember 7, 2009 Rick McGee, PhD Lew Smith, MD Bill Schnaper, MD
What we know for sure… • The review criteria and scoring system changed last spring – used in Review Groups and Challenge Grants since then • In theory, designed to put more weight on Impact – importance of the work • Review criteria are changed in subtle ways, although not clear how much reviewers see or are responding to them • Page lengths for most submission are substantially changed as of January 25, 2010 • No apparent major changes in emphasis on content and review of K awards – Lew – Confirm?
What we don’t know… • Much more than what we do know • How reviewers will or will not change • How much weight will really be placed on ‘impact’ and how that will be defined by reviewers • How to say in 13 pages what we have barely been able to say in 25 pages for an R01 • How applications with mathematically identical scores will be differentiated and by whom • How the new paradigm will impact beginning, mid and late career scientists • BUT – most of these you can’t control!
Comparison of Old and Enhanced Criteria • See handout with Orange banner • Overall Impact – the score that matters • Core Review Criteria – old vs. new • Significance Significance • Approach Investigator(s) • Innovation Innovation • Investigator(s) Approach • Environment Environment
Significance – changes… • “…important problem or a critical barrier to progress in the field?” • “If the aims achieved…scientific knowledge, technical capability, and/or clinical practice…” • “How will successful completion of the aims change (vs. affect) ….this field?”
Investigator(s) – changes… • “Are the PD/PIs, collaborators and other researchers (not key personnel) well suited (not trained) to carry out the project?” • “If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)?” • (for multi-PD-PI) “Are their leadership approaches, governance and organizational structure appropriate for the project?”
Innovation – changes… • Read the two side by side – hard to tell if they are different or the same…
Approach – changes… • “Are potential problems, alternative strategies, and benchmarks for success presented?” vs • “Does the applicant acknowledge potential problems areas and consider alternative tactics? • Issues related to multi-PI and administrative/ management of project removed • New paragraph on clinical research added
Environment – changes… • “Will” replaces “Does” as the first word • “Are the institutional support, equipment, and other physical resources available to the investigators adequate for the project proposed?” – more detailed than before
New Scoring System/Process • Reviewers read assigned grants at home • Submit reviews and initial scores before meeting • Preliminary decisions of which grants to review made based on preliminary scores • Final decision of grants to review made at start of meeting • Review and final scoring as before, although no requirement to change scores on individual criteria to match final impact score • Impact Score does not have to align with individual criteria in any predictable way
New Scoring System/Process • See handout “Scoring System and Procedure” page 4 for score descriptors • Percentiling also addressed there • Bulleted list of reviewer comments on strengths and weakness for each criterion replaces narratives
Page Limit Changes • See handout #3 – PHS SF424 (R&R) pages I-22 – I-24 • R02, R13, R21 – 1 page Specific Aims plus 6 page Research Plan • R01 and some others – 1 page Specific Aims plus 12 page Research Plan • K08 and K23 – 12 pages for Candidate Information and Research Strategy
Possible Implications – Speculation… • Specific Aims page will continue to be critical to first impressions • Overall writing style must be very compact and crisp – no wasted words! • Seems to be less focus on Background – very targeted historical perspective • Preliminary data will have to be streamlined if you have a lot • Carefully choose the details presented in the experimental design – potentially broader brush than in the past
Possible Implications – More Speculation… • Early career investigators continue to be treated differently • Established investigators appear to be evaluated more on long-term track record • Increasingly important to make Impact/novelty/innovation very obvious but it must be legitimate! – Critical to the field if not a direct health impact