1 / 16

Goals of this Video

ENHANCING PEER REVIEW What Reviewers Need to Know Now Slides Accompanying Video of Dr. Alan Willard, March 2009 . Goals of this Video. Prepare for the May/June Review Meetings Supplement resources on the Enhancing Peer Review Web site: http://enhancing-peer-review.nih.gov

london
Download Presentation

Goals of this Video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ENHANCING PEER REVIEWWhat Reviewers Need to Know Now Slides Accompanying Video of Dr. Alan Willard, March 2009

  2. Goals of this Video • Prepare for the May/June Review Meetings • Supplement resources on the Enhancing Peer Review Web site: http://enhancing-peer-review.nih.gov • Video – Overview of Enhancements • Frequently Asked Questions • E-mailbox for Questions

  3. Changes to ReviewBeginning at May/June 2009 Meetings • Enhanced Review Criteria • Significance, Investigator(s), Innovation, Approach, Environment • Templates for Structured Critiques • Scoring of Individual Review Criteria • New 1-9 Scoring Scale

  4. Goals of the Changes • Clearer understanding of the basis of application ratings • More emphasis on impact and less emphasis on technical details • Succinct, well-focused critiques that evaluate, rather than describe, applications • Routine use of the entire rating scale

  5. Before the Review Meeting When reading applications the assigned reviewers should: • Address all applicable criteria and other review considerations • Identify major strengths and weaknesses • Assign scores to each of the 5 “core” criteria • Assign an overall impact/priority score

  6. Preparation of Critiques When writing your critiques: • Use bulleted points to make succinct, focused comments • Short narratives may occasionally be appropriate, but should be rare • Focus on major strengths and weaknesses (ones that impacted your overall rating of the application)

  7. Features of Critique Templates • Boxes for evaluating: • Each core review criterion • Other applicable review criteria and considerations • Overall impact of the application • A box for “advice to applicants” • Hyperlinks to web pages providing descriptions of review criteria and additional review considerations

  8. Excerpt from a Critique Template • List major strengths and weaknesses that influenced the overall impact/priority score • Limit text to ¼ page per criterion, although more text may occasionally be needed

  9. Scoring of Individual Review Criteria • There are 5 “core” criteria for most types of grant applications • For example, the core criteria for R01s are: • Significance • Investigator(s) • Innovation • Approach • Environment • Use the 9-point scale (1 = exceptional, 9 = poor)for the five “core” review criteria.

  10. Overall Impact/Priority Scores • Consider criterion strengths and weaknesses of each application in determining an overall impact/priority score • Recognize this is a NEW scoring system and focus on the guidelines for its use • This new scoring system is intended to reflect the “real-world” range of the quality of applications typically seen in actual study sections • It is ESSENTIAL that reviewers take advantage of this unique opportunity to use the entire 1-9 range

  11. Scoring Descriptions

  12. Before Attending the Review Meeting • Post critiques to the Internet-Assisted Review (IAR) Web site • Enter criterion scores and overall/priority score in IAR • IAR tutorials and documentation are available on the Enhancing Peer Review Web site

  13. Procedure for Discussed Applications • Assigned reviewers will discuss strengths and weaknesses of each application • Recommend overall impact/priority score • Criterion scores will not be discussed by the committee • All eligible members will record an overall impact/priority score (as is presently true)

  14. After the Review Meeting To justify the overall impact/priority score: • Assigned reviewers whose opinions changed as a result of discussion at the meeting should use IAR: • To modify their criterion scores • To post revised critiques

  15. Summary Statements • Overall impact/priority scores of discussed applications will be the mean of scores voted by all eligible reviewers, multiplied by 10 • Final scores will range from 10-90, in whole numbers • Summary statements for ALL applications will include the criterion scores and critiques posted by assigned reviewers

  16. For additional information: Enhancing Peer Review at NIH Web Site http://enhancing-peer-review.nih.gov Thank you for your review service

More Related