1 / 28

Archived File

Archived File. The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files. Andrea Kopstein, PhD.

alena
Download Presentation

Archived File

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.

  2. Andrea Kopstein, PhD Evaluation of CSR Peer Review Pilots PRAC - February 1, 2010 National Institutes of HealthU.S. Department of Health and HumanServices

  3. American Recovery & Reinvestment Act

  4. ARRA INITIATIVES • Challenge Grants (RC1) • Competitive Revisions • Grand Opportunities (“GO” Grants) • High End Instrumentation • Small Business • Academic Research EnhancementAward (AREA) • Community Infrastructure

  5. Applications Reviewed and Reviewers Used by CSR in June 2008 and 2009

  6. The ARRA Reviewers • Challenge review used “editorial board” format – two stages • Survey - 14,683 ARRA reviewers invited to participate • 7,653 individuals completed on-line survey (response rate of 52%) • 6,548 mail reviewers (Stage 1) • 1,105 in-person reviewers (Stage 2)

  7. ARRA Reviewer Experience

  8. Where ARRA Reviewers Work

  9. Academic Rank ARRA & Non ARRA Reviewers

  10. Shorter Applications had Enough Information …

  11. Impact of Application Size

  12. Order of Review/Clustering Pilot

  13. Order of Review/Clustering Pilot • February-March 2009 • Evaluation invited 663 reviewers and 29 SROs • Pilot included review of 1,628 applications • 415 reviewers and 25 SROs responded • Response rate: 63% percent of reviewers, and 86% of the SROs

  14. Preliminary rank order of review Discussion of applications in order (“best to worst”) of average preliminary impact score from assigned reviewers: Requirement: • Reviewers participate in entire meeting. • Discourages telephone reviewers for 1 or 2 applications

  15. Satisfaction with Review in Preliminary Score Order

  16. Clustering Pilot • Clustering: “like” applications grouped together for peer review (such as clinical applications, by grant activity (e.g. R01, R21 etc.) • “new investigator/early stage investigator” (NI/ESI) – only for R01s

  17. Satisfaction with Peer Review that Includes Clustering

  18. Was the Clustered Review Process Fair to NI/ESI Applicants?

  19. Transformative R01 Initiative

  20. Transformative Research Projects Program - T-R01s • “support exceptionally innovative, high risk, original, and/or unconventional research with the potential to create new or challenge existing scientific paradigms.” • “Editorial board” format – three stages • Evaluation - stage 1/3 reviewers and applicants (prior to review) • 431/700 applicants and 7/11 reviewers responded, 62% for applicants & 63% for reviewers.

  21. T-R01 • 740 applications received • 720 reviewed (3 stage) • 42 funded ($32 million) • 431 applicants and 7 reviewers responded, response rates of 62% for applicants and 63% for reviewers.

  22. T-R01 Applicants • Majority Caucasian and non-Hispanic: • Caucasian: 71%; non-Hispanic/Latino: 86%; Asian: 21%; African Americans: 2%; American Indians: <1% • Majority male (79% men, 21% women)

  23. T-R01 Applicant Age, N = 431

  24. Applicants – funding sources

  25. Applicant Findings • Two commonly occurring research areas: molecular/cellular/chemical biologists and clinical/translational research • 80% of applicants claimed proposed concept was a significant departure from usual research

  26. Applicants – Possibility of Receiving Funding From other Sources

  27. Reviewer opinions of mail reviewers having appropriate expertise …….

  28. T-R01 Reviewers • 50% of applicants understood project goals • 25% of applicants capable of conducting transformative research

More Related