Disclosure I do not have any relationship(s) with commercial interests.

1. LSU School of Medicine-New Orleans (LSUSOM-NO) is the provider of Continuing Medical Education for this activity. The planning and presentation of all LSUSOM-NO activities ensure balance, independence, objectivity and scientific rigor. The LSU School of Medicine-New Orleans designates this educational activity for a maximum of 1.5 AMA PRA Category 1 Credit(s) ™. Physicians should claim only the credit commensurate with the extent of their participation in the activity. LSUHSC-NO ORS Training Series

2. Disclosure Idonothave any relationship(s) with commercial interests. A commercial interest is any entity producing, marketing, re-selling, or distributing health care goods or services consumed by, or used on, patients. LSUHSC-NO ORS Training Series

3. The NIH Peer Review Process LSUHSC-NO ORS Training Series

4. Welcome • Presenter: • Nicole G. Hammill, MBA • Coordinator of Grants and Development • Office of Research Services • 433 Bolivar Street, Room 206E, New Orleans, LA 70112 • Nbarro@lsuhsc.edu • (504) 568-4970 tel • (504) 568-8808 fax • http: http://www.lsuhsc.edu/administration/academic/ors/grants_contracts_processing.aspx LSUHSC-NO ORS Training Series

5. Office of Research Services Director: Dr. Kenneth E. Kratz Staff: Nicole G. Hammill – Pre-award (Grants and Contracts) Rose Castay – IACUC and IBC Dyan Melson – IRB Lynn Arnold – IRB Amy Tassin – IRB Anissa McDougle – Conflicts of Interest Responsibilities: • Pre-award, sponsored project activity; this includes evaluation and routing for signatures all grant applications, research agreements, and clinical trial agreements. • Conflict of Interest Program based upon Chancellor’s Memorandum #35 “Individual and Institutional COI in Sponsored Projects”. • The AAHRPP “Fully Accredited” Human Research Protection Program and Institutional Review Board (IRB) which provides oversight for the protection of human subjects participating in research. • The Institutional Animal Care and Use Committee (IACUC) which provides oversight for the welfare of animals used in research. • The Institutional Bio-safety Committee (IBC) which in collaboration with the Office of Environmental Health and Safety provides oversight of bio-safety issues and recombinant DNA research. LSUHSC-NO ORS Training Series

6. Helpful Administrative Information • Most of the numbers, dates, names, and titles commonly needed for the completion of grant applications can be found here: http://www.lsuhsc.edu/administration/academic/ors/docs/Helpful_Administrative_%20Information.pdf Updated frequently! LSUHSC-NO ORS Training Series

7. Two-Tiered Process • Mandated by law – PHS Health Act • Defined in federal regulation – 42 CFR 52h • Further defined in NIH policy • Per year: • Nearly 80,000 applications • Over 18,000 reviewers LSUHSC-NO ORS Training Series

8. Two-Tiered Process • First Tier: Initial peer review • Recommendations on scientific and technical merit • Scientific Review Groups (“SRGs” or “Study Sections”) • Second Tier: Advisory Council or Board • Recommendations to the Institutes/Centers (“ICs”) on funding, appeals, program priorities • “Council” • Final funding decisions – IC Director LSUHSC-NO ORS Training Series

9. Referral of Application • Application is received at the NIH Center for Scientific Review (“CSR”). • First Tier assignments include: • Scientific Review Group (SRG) • Study Section • Institute/Center (IC) • Scientific Review Officer • Second Tier assignments include: • Advisory Council or Board LSUHSC-NO ORS Training Series

10. Referral of Application, con’t. • Funding considerations include: • Specific ICs (each IC has its own budget, priorities, and paylines) • Dual assignment (to more than one IC) is possible • Input from Program Officer • Funding decisions • Ultimately made by the director of the IC LSUHSC-NO ORS Training Series

11. Referral to a Scientific Review Group CSR Review • Most R01s, fellowships, and small business applications • Some Program Announcements (PAs, PARs) • Some Requests for Applications (RFAs) IC Review • IC-specific features • Program projects • Training grants • Career development awards • RFAs The review locus is stated in the Funding Opportunity Announcement (“FOA”) LSUHSC-NO ORS Training Series

12. Sample Review Loci LSUHSC-NO ORS Training Series

13. Requesting Referral to a Specific SRG • Include in the application’s Cover letter: • Application title • FOA # and title • Request: • Assignment to particular SRG or study section • SRG rosters are posted 30 days before the SRG meeting: • http://era.nih.gov/roster/index.cfm • http://www.csr.nih.gov/committees/rosterindex.asp • Assignment to particular IC for funding consideration • Provide: • Disciplines involved, if multidisciplinary • Explanation for late application, if necessary Not all requests can be honored LSUHSC-NO ORS Training Series

14. The Scientific Review Officer (SRO) • First level of peer review • Designated Federal Official • Extramural scientist administrator • Identifies and recruits reviewers • Manages conflicts of interest • Oversees arrangements for review meetings • Presides at review committee meetings • Prepares and releases summary statements LSUHSC-NO ORS Training Series

15. Peer Reviewers • Recruiting Criteria include: • Expertise • Stature in field • Mature judgment • Impartiality • Ability to work well in a group • Managed conflicts of interest • NIH attempts to ensure: • Balanced representation • Gender • Geography • Diversity • Seniority • Availability LSUHSC-NO ORS Training Series

16. Types of Scientific Review Groups (SRGs) • “Chartered” SRGs • Multi-year terms • Formal appointment process • May include temporary members for special expertise • Special Emphasis Panels (SEPs) • Ad hoc membership • Often meet only once LSUHSC-NO ORS Training Series

17. Types of Reviewers • Regular reviewers – permanent and temporary • Preliminary impact/priority scores, criterion scores, written critiques • Final impact/priority scores • Other Contributing Reviewers (“mail” reviewers) • Written critiques, criterion scores, preliminary impact/priority scores • Cannot submit final impact/priority scores LSUHSC-NO ORS Training Series

18. Reviewer Assignments • For each application: • ≥ Three qualified reviewers are assigned (“2 + 1”) • Assignments are made by the SRO • Based on the scientific content of application • Expertise of the reviewer • Suggestions from the PI on types of expertise – • not names! • Suggestions from Program staff • Suggestions from SRG members • Managing conflicts of interest • Balancing workload • Assignments are confidential LSUHSC-NO ORS Training Series

19. Reviewer Conflicts of Interest (COI) • Potential COIs between a reviewer and an application: • Financial • Employment • Personal • Professional • SRG membership • Other interests Two COI vouchers are submitted by each SRG member. LSUHSC-NO ORS Training Series

20. Scientific Review Groups (SRGs) • Make recommendations on merit - not funding! • Scientific and technical merit • Budget and project duration • Protection of human subjects, inclusion plans, vertebrate animals, biohazards • Resource Sharing Plans • Other administrative factors • Provide: • Impact/priority scores • Criterion scores • Written critiques LSUHSC-NO ORS Training Series

21. Confidentiality • All confidential materials, discussions, documents are deleted, retrieved, or destroyed • Reviewers sent guidance with applications • Application information provided on secure websites or protected portable devices • All questions must be referred to SRO • SRG meetings are closed to the public • Program staff may observe SRG meeting Do not contact reviewers directly! LSUHSC-NO ORS Training Series

22. Application Scoring • Overall Impact: • Likelihood for the project to exert a sustained, powerful influence on the research field(s) involved • In consideration of: • At least five scored criteria • Receive individual, numerical scores • Additional criteria in certain announcements • Additional review criteria • As applicable for the project proposed • Do not receive individual, numerical scores • Additional criteria in certain announcements LSUHSC-NO ORS Training Series

23. Scored Review Criteria • Applications for: • Research Grants • Cooperative Agreements • Significance • Investigator(s) • Innovation • Approach • Environment • (FOA-specific criteria) • Other review criteria apply to other mechanisms • See “Review Criteria at a Glance” (http://grants.nih.gov/grants/peer/reviewer_guidelines.htm) LSUHSC-NO ORS Training Series

24. Review Criterion: Significance • Does the project address an important problem or a critical barrier to progress in the field? • If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? • How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field? LSUHSC-NO ORS Training Series

25. Review Criterion: Investigator(s) • Are the PD/PIs, collaborators, and other researchers well suited to the project? • If Early Stage Investigators or New Investigators, or in the early stages of independent careers, do they have appropriate experience and training? • If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? • If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project? LSUHSC-NO ORS Training Series

26. Review Criterion: Innovation • Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? • Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? • Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed? LSUHSC-NO ORS Training Series

27. Review Criterion: Approach • Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? • Are potential problems, alternative strategies, and benchmarks for success presented? • If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed? LSUHSC-NO ORS Training Series

28. Review Criterion: Approach (con’t.) • If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed? LSUHSC-NO ORS Training Series

29. Review Criterion: Environment • Will the scientific environment in which the work will be done contribute to the probability of success? • Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? • Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements? LSUHSC-NO ORS Training Series

30. Additional Review Criteria • As applicable for the project proposed, reviewers: • Consider in determining scientific and technical merit • Do not give separate scores for these items • FOA-specific criteria • Protections for Human Subjects • Inclusion of Women, Minorities, and Children • Vertebrate Animals • Resubmission Applications • Renewal Applications • Revision Applications • Biohazards LSUHSC-NO ORS Training Series

31. Additional Review Considerations • As applicable for the project proposed, reviewers: • Address each item • Do not give scores for these items • Should not consider them in providing an overall impact/priority score • FOA-specific considerations • Select Agent Research • Applications from Foreign Organizations • Resource Sharing Plans • Budget and Period Support LSUHSC-NO ORS Training Series

32. NIH Scoring System • Numerical scores • 1.0 (exceptional) to 9.0 (poor) • Final impact/priority score - average of individual scores x 10 • Individual criterion scores • Ranked by percentile for certain mechanisms • Not Discussed (ND) – streamlining • Other designations (DF = Deferred, AB = Abstention, CF = Conflict, NP = Not Present) • Final impact/priority scores range from 10 through 90. LSUHSC-NO ORS Training Series

33. NIH Scoring System (con’t) • Preliminary scores (before the SRG meeting) • Entered by assigned reviewers and discussants in secure website • Made available to other SRG members • Final overall impact/priority scores (at the SRG meeting) • Voted by private ballot • All eligible SRG members vote • Reviewers are instructed to revise their criterion scores after the meeting. LSUHSC-NO ORS Training Series

34. Score Descriptors LSUHSC-NO ORS Training Series

35. Streamlining • Allows discussion of more meritorious applications • Less meritorious applications tabled at the SRG meeting, designated Not Discussed (ND) • Requires full concurrence of the entire SRG • Summary statement: • Reviewer critiques • Individual criterion scores • No numerical, overall impact/priority score LSUHSC-NO ORS Training Series

36. Streamlining (con’t) • Score order of review • SRG discusses most meritorious applications first • Entire SRG decides when to stop, which applications will not be discussed in panel • Other order of review (e.g., IC assignment, mechanism) • SRO prepares a list of average preliminary scores • Distributes to SRG • Entire SRG decides which applications to discuss LSUHSC-NO ORS Training Series

37. Pre-Meeting SRG Procedures • SRO • Performs administrative review of applications • Recruits reviewers, arranges for meeting date and site • Assigns 3 SRG members to each application • Makes applications available to reviewers • Internet Assisted Review (IAR) site or on CDs • Usually about six weeks before the SRG meeting • Instructs reviewers in review procedures • Monitors posting of initial scores and critiques in IAR • Documents for Reviewers are available at: http://grants.nih.gov/grants/peer/reviewer_guidelines.htm#general_guidelines LSUHSC-NO ORS Training Series

38. Structured Critiques • New summary statement format • Bulleted comments from reviewers, less text • Criterion scores from assigned reviewers • Decreases variability • Increases quality of information in critiques • More succinct, better organized • Encourages evaluative statements • Ensures that reviewers address all review criteria and considerations • Reviewers also write a paragraph summarizing the factors that informed their overall impact score to supplement the bulleted critiques. • Critique templatesare availableat: http://grants.nih.gov/grants/peer/reviewer_guidelines.htm#general_guidelines LSUHSC-NO ORS Training Series

39. Templates for Reviewers LSUHSC-NO ORS Training Series

40. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

41. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

42. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

43. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

44. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

45. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

46. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

47. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

48. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

49. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series

50. Templates for Reviewers (con’t) LSUHSC-NO ORS Training Series