An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director,

1. An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association for Thoracic Surgery March 4, 2011 National Institutes of HealthU.S. Department of Health and Human Services

2. National Institutes of Health Office of the Director National Institute on Aging National Institute on Alcohol Abuse and Alcoholism National Institute of Allergy and Infectious Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute of Dental and Craniofacial Research National Institute of Diabetes and Digestive and Kidney Diseases National Institute on Drug Abuse National Institute of Environmental Health Sciences National Eye Institute National Institute of General Medical Sciences National Heart, Lung, and Blood Institute National Human Genome Research Institute National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Nursing Research National Institute of Biomedical Imaging and Bioengineering National Center for Complementary and Alternative Medicine National Library of Medicine National Center on Minority Health and Health Disparities Fogarty International Center National Center for Research Resources Center for Scientific Review Clinical Center Center for Information Technology

3. Review Process for a Research Grant National Institutes of Health Research Grant Application School or Research Center Center for Scientific Review Assigns to IC & I IRG/Study Section Study Section Initiates Research Idea Submits Application Reviews for Scientific Merit Institute Evaluates for Relevance Allocates Funds Advisory Councils and Boards Conducts Research Recommends Action Institute Director Takes Final Action

4. Scientific Review Process Dual Review System for Grant Applications First Level of Review CSR or Institute Review Scientific Review Group (Study Section) Second Level of Review NIH Institute/Center Council

5. Your Application Goes to the NIH Center for Scientific Review (CSR) • Receives all NIH applications • Refers them to NIH Institutes/Centers and to scientific review groups • Reviews majority of grant applications for scientific merit Focal Point for Initial Review at NIH

6. AIDS, Behavioral and Population Sciences Basic and Integrative Biological Sciences Physiological and Pathological Sciences Translational and Clinical Sciences Neuroscience, Development and Aging Endocrinology, Metabolism, Nutrition & Reproductive Sciences Biobehavioral & Behavioral Processes Biological Chemistry & Macromolecular Biophysics Cardiovascular and Respiratory Sciences Brain Disorders & Clinical Neuroscience Risk, Prevention& Health Behaviors Surgical Sciences, Biomedical Imaging and Bioengineering Molecular, Cellular & Developmental Neuroscience Bioengineering Sciences & Technologies Immunology Epidemiology & Population Sciences Integrative, Functional & Cognitive Neuroscience Cell Biology Infectious Diseases & Microbiology Musculoskeletal, Oral And Skin Sciences Healthcare Delivery & Methodologies Genes, Genomes & Genetics Emerging Technologies & Training in Neuroscience Oncology 2 – Translational Clinical Digestive, Kidney & Urological Systems AIDS & Related Research Oncology 1 – Basic Translational Vascular and Hematology Biology of Development and Aging Interdisciplinary Molecular & Training CSR Review Divisions

7. Help Get Your Application to the Right Study Section • Review CSR Integrated Review Group and Scientific Review Group (Study Section) guidelines to identify a home for your application. • Submit a Cover Letter!

8. Help Get Your Application to the Right Study Section http://www.csr.nih.gov/

9. Help Get Your Application to the Right Study Section Integrated Review Group

10. Help Get Your Application to the Right Study Section Study Section

11. Cardiovascular and Respiratory Sciences IRG • Cardiac Contractility, Hypertrophy and Failure • Cardiovascular Differentiation and Development (CDD) • Clinical and Integrative Cardiovascular Sciences (CICS) • Electrical Signaling, Ion Transport and Arrhythmias (ESTA) • Myocardial Ischemia and Metabolism (MIM) • Lung Injury, Repair and Remodeling (LIRR) • Lung Cellular, Molecular and Immunobiology • Respiratory Integrative Biology and Translational Research (RIBT)

12. Surgical, Biomedical Imaging and Bioengineering IRG • Clinical, Molecular Imaging and Probe Development (CMIP) • Medical Imaging (MEDI) • Surgery, Anesthesiology and Trauma (SAT) • Bioengineering, Technology and Surgical Sciences (BTSS) • Biomedical Imaging Technology A & B (BMIT)

13. Division of Translational and Clinical Sciences (DTCS) Trans-IRG Imaging SEP • Goal - enhance review of multidisciplinary, translational research with a focus on imaging • Transformative in nature (translational research) • Include humans • Use imaging as a measure of diagnosis, intervention or treatment success/efficacy • Editorial Board (Two Stage) Review • First stage – 3 mail reviews (technical focus) • Second stage – 3 clinical impact reviews • Crosses disciplines of cancer, cardiovascular, musculoskeletal and neuroscience

14. Submit a Cover Letter The cover letter should be used for a number of important purposes: • Suggest Institute/Center assignment • Suggest review assignment • Identify individuals in conflict • Identify areas of expertise needed to evaluate the application • Discuss any special situations It is NOT appropriate to use the cover letter to suggest specific reviewers.

15. Peer Review in CSR • CSR Study Sections are managed by a Scientific Review Officer (SRO) who is a doctoral-level professional, whose scientific background is close to the focus of the study section. • Each CSR standing study section has 12-25 regular members who are from the scientific community. • Temporary members are recruited as needed. • About 60-100 applications are normally reviewed at each study section meeting.

16. The Role of Your Scientific Review Officer Designated Federal Official responsible for the overall review process

17. Your Study Section Meeting

18. Your Application Is Reviewed In Order • Clustered and Ordered for Fairness • New and Early Stage Investigator applications are usually reviewed first • Applications in each cluster are reviewed in order of their preliminary scores

19. New Investigator (NI): • PD/PI who has not yet competed successfully for a substantial NIH research grant • For multiple PD/PIs-all PD/PIs must meet requirements for NI status • Early Stage Investigator (ESI): • PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)

20. Discussions Focus on the Best Applications • Reviewers discuss about 50-60% of the applications • The panel will discuss any application a reviewer wants to discuss

21. Additional Review Platforms Helps to recruit reviewers Electronic review modes reduce travel Electronic Reviews Telephone Assisted Meetings Video Assisted Meetings Telepresence Internet Assisted Meetings Editorial Board Review – multidisciplinary

22. Review Criteria • Overall Impact: • Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved • Core Criteria • Significance • Investigator(s) • Innovation • Approach • Environment • Review criteria each scored from 1-9

23. How Your Application Is Aligned to the Review

24. 9-Point Scoring Scale

25. Your Summary Statement • Scores for each review criterion • Essentially unedited critiques • Administrative notes if any If your application is discussed, you also will receive: • An overall impact/priority score and percentile ranking • An Summary of review discussion • Budget recommendations

26. Program Officer Summary Statement Impact/Priority Score 10-90 range Percentile in whole numbers Percentile: 29 New Indicator for Early Stage Investigators

27. When Preparing an Application • Read instructions • Never assume that reviewers will know what you mean • Refer to literature thoroughly • State rationale of proposed investigation • Include well-designed tables and figures • Present an organized, lucid write-up • Obtain pre-review from faculty at your institution NIH Grant Writing Tips http://grants.nih.gov/grants/grant_tips.htm

28. Additional Factors to Consider • Focus on significance/impact • Make it exciting • Be very clear • Do not assume too much • Have realistic aims and timelines -- Don’t be too ambitious • Be brief with things that everybody knows • Note the study’s limitations • Proofread the application

29. Who Can Answer Your Questions? Before You Submit Your Application • A Program Officer at an NIH Institute or Center After You Submit • Your Scientific Review Officer After Your Review • Your Assigned Program Officer

30. Key NIH Review and Grants Web Sites NIH Center for Scientific Review http://www.csr.nih.gov NIH Office of Extramural Research http://grants.nih.gov/

31. Helpful Handouts Insiders Guide What Happens to NIH Grant Application to Peer Review Your Grant Application Useful Web Links http://cms.csr.nih.gov/publications/