Office of Extramural Research, Education and Priority Populations (OEREP) Grantsmanship Session

1. Office of Extramural Research, Education and Priority Populations (OEREP)Grantsmanship Session Overview: Francis Chesley, MD – OEREP Director Prepare a Grant Application: Debbie Rothstein, PhD, Senior Advisor to the Director - OEREP AHRQ Peer Review Process: Kishena Wadhwani, PhD, MPH – Div. Scientific Review Director, OEREP Human Subjects Protections and Inclusions: Patrick McNeilly, PhD – AHRQ IRB Official, OEREP

2. Preparing a Grant Application: Steps to Success Debbie Rothstein, Ph.D. Senior Advisor, Extramural Research Office of Extramural Research, Education and Priority Populations Agency for Healthcare Research and Quality September 9, 2008

3. Getting Started • Internet is funding gateway • AHRQ and NIH use essentially the same application processes and grant mechanisms • Keep abreast of open solicitations and Agency research priorities

4. AHRQ’s Website – www.ahrq.gov

5. AHRQ and NIH: Grant Application Similarities • Application forms: SF 424(R&R), PHS 398 • Application submission dates • Use of internet as the funding gateway to keep abreast of research priorities and open solicitations, staff can provide technical assistance • Announcement publication: NIHGuide for Grants: http://grants1.nih.gov/grants/guide/index.htmlandGrants.Gov:www.http://grants.gov

6. AHRQ and NIH: Grant Application Similarities (cont’d) • Receipt and referral of grant applications • Application review procedures • Grant mechanisms – e.g., R01, R03, R13, R36, K01, K02, K08 • Transition to electronic submission of applications • Some common Policy Notices and Funding Opportunity Announcements

7. AHRQ and NIH: Grant Application Differences • Detailed budget vs. modular budget • Conference grants – AHRQ has small and large opportunities • AHRQ doesn’t allow multiple PI’s • Budget limitations • Large grants (R01, R18) - $300K vs. $500K • Small grants (R03) - $100K vs. $50K • Fewer grant mechanisms used at AHRQ

8. AHRQ Grant Opportunities • AHRQ announces availability of grants in a Funding Opportunity Announcement (FOA) • FOA’s are published in the NIH Guide for Grants (http://grants.nih.gov/grants/guide/index.html) and posted on Grants.gov (http://www.grants.gov/ • Program Announcement (PA) • Describes broad research interests • Open over a period of time • Request for Applications (RFA) • More well defined area, specific program objectives • Single application receipt date and amount of funds

9. Common AHRQ Grant Opportunities • R01: large research grants • Feb 5, June 5, Oct 5 application receipt dates • up to $300K/yr • R18: research demonstration grants • Jan 25, May 25, Sept 25 application receipt dates • up to $300K/yr • R03: small research projects • Feb 16, June 16, Oct 16 application receipt dates • up to $100K in total costs • Large and small conference grants (R13) • Large: Apr 12, Aug 12, Dec 12 receipt; up to $100K/yr in total costs • Small: Feb 20, Apr 20, Jun 20, Aug 20, Oct 20, Dec 20 receipt; up to $50K

10. AHRQ Grant Opportunities: Training/Career Development • Pre and Postdoctoral Training • Institutional Training Programs (T32) • Individual Pre-doctoral Fellowships to Promote Diversity (F31) • Individual Post-doctoral Fellowships (F32) • Dissertation Grants (R36) • Career Development Awards • Mentored Scientist Awards (K01) • Independent Scientist Awards (K02) • Mentored Clinical Scientist Awards (K08) • Detailed information at: http://www.ahrq.gov/fund/training/trainix.htm • Send questions to: training@ahrq.hhs.gov

11. AHRQ Staff Involved in Grants Process • Referral Officer • Review Staff – Scientific Review Officer and Grants Management Specialist • Program Staff – Project Officer (PO) • Grants Management Staff

12. Electronic Application and Submission • Transition to Electronic Receipt began Dec, 2005 • Vast majority of all competing applications to AHRQ are now submitted electronically using SF 424 (R&R) application • All major mechanisms used by AHRQ have transitioned except for the training grants

13. What is Grants.gov? • Federal government’s single, online portal to electronically: • Find Grant Opportunities • Apply for Grants • A cross-agency initiative spanning 900 grant programs from the 26 grant-making agencies, and over $350 billion in annual awards.

14. Electronic Submission of Grant Applications • With this new process, it’s critical to read and follow all instructions in the announcement and application guide • Applicant must prepare for e-submission by completing mandatory registrations • Grants.gov: registration for organization • ERA Commons: registration for organization and the PI

15. SF424 (R&R) Application Form • Provides a consistent electronic submission process through Grants.gov • Consists of common data elements, arranged in components • Not all components will be used for every Funding Opportunity Announcement (FOA) • Each FOA will have the appropriateapplication package attached • From data files, application image is generated • Applicant should view and verify the application image

16. Electronic Application and Submission - General Information and Help Links • SF424 (R&R) Application and Electronic SubmissionInformation: http://grants.nih.gov/grants/funding/424/index.htm • General information on Electronic Submission of Grant Applications: http://era.nih.gov/ElectronicReceipt/ • Finding Help: http://era.nih.gov/ElectronicReceipt/support.htm

17. Paper No More, Use 424 (R&R)OLD NEW

18. What Determines Which Awards Are Made? • Scientific merit • Significance and originality • Methods • Program / Agency considerations • What is uniquely AHRQ • Existing research portfolio balance • Anticipated IMPACT of research • Availability of funds

19. Ingredients of a Successful Grant Application • Good Idea • Good Science • Good Application • Fits Agency Research Priorities

20. AHRQ PEER REVIEW PROCESS Kishena C. Wadhwani, Ph.D., M.P.H. Director, Division of Scientific Review (DSR) Kishena.wadhwani@ahrq.hhs.gov Phone: (301) 427-1556 Office of Extramural Research, Education and Priority Population (OEREP) Agency for HealthCare Research and Quality (AHRQ) Department of Health and Human Services (DHHS) 10/09/2008

21. ORGANIZATIONAL CHART Office of Extramural Research, Education and Priority Populations (OEREP) Division of Scientific Review (DSR) Current Study Review Group (SRG) Clusters and the Corresponding SRO/GMS OEREP Director: Dr. Francis Chesley Committee Management Receipts & Referrals DSR Director: Dr. Kishena Wadhwani Study Section Clusters Health CareSystem Research (HSR) Health Care Quality & Effectiveness Research (HCQER) Health Care Technology and Decision Sciences (HCTDS) Health Care Research Training (HCRT) SRO: Dr. Carl Ohata GMS: Maureen Gallagher SRO: TBD GMS: Ms. Carmen Colston SRO: Dr. Boris Aponte GMS: Ms. Rebecca Trocki SRO: Ali Azadegan GMS: Ms. Diane Manuel

22. AHRQ HEALTH SERVICES RESEARCH SRG AND ITS PRIMARY RESEARCH FOCI

23. SCHEDULE OF STUDY SECTION MEETINGS, FY08-FY09 (TENTATIVE) (Updated: 09/01/2008)

24. Process of Review for New Individual Research Project Grant Applications • Three overlapping cycles per year: • Submit in February(June, October) • - Review in June –July(Oct- Nov,February - March) • SLT in July - August(December,May - June) • Earliest award in December(April,August - September)

25. Peer Review at NIH/AHRQ: Types of Committees (Study Sections): • Standing Committees - Chartered; multi-year commitment - Temporary members added as needed - Roster posted on NIH/AHRQ websites • Special Emphasis Panels (SEPs) - All Temporary (Ad-hoc) members

26. Purpose of Peer Review To evaluate the scientific and technical merit of grant applications, providing information(Recommendations) used by the Agencies/Institutes and Centers to make funding decisions.

27. Reviewers consider • Scientific and technical merit of proposed research/training • Qualifications of Principal Investigator(s) and research team • Availability of resources (Labs, Institutions)

28. Reviewers also consider (Cont.) • Reasonableness of requested budget for work proposed • Other factors (e.g., human subjects, animal welfare, inclusion policies/plans)

29. Reviewers do not consider • Program relevance (Except if stated in FOA) • Policy issues • Funding levels • Anticipated budget reductions • Comparisons with other applications

30. Review Criteria • For all R01, R03, R15, R21, and P01 subprojects: • Significance and Originality • Approach – Methods and Data • Innovation • Organization of the Project • Investigators • Environment – Facilities and Resources

31. Sole Basis of Review: Review Criteria (The NIH 5 – as an example) Significance:Does this study address an important problem? If the aims of the application are achieved, how will scientific knowledge be advanced? What will be the effect of these studies on the concepts or methods that drive this field? Approach:Are the conceptual framework, design, methods, and analyses adequately developed, well-integrated, and appropriate to the aims of the project? Does the applicant acknowledge potential problem areas and consider alternativetactics? Innovation:Does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies? Investigator:Is the investigator appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the Principal Investigator and other researchers (if any)? Environment: Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of the unique features of the scientific environment or employ useful collaborative arrangements? Is there evidence of institutional support?

32. Also... • Reviewers evaluate • Protection of human subjects, the environment, and animal welfare • Inclusion of women, minorities, and children • Inclusion of AHRQ Priority Populations • Budget

33. Research Involving Human Subjects Important Considerations that must be addressed in the application because they impact on priority score - considered to be part of the Approach • Are there any risks* to the human subjects? • Are the protections adequate? • Are there potential benefits to the subjects and to others? • What is the importance of the knowledge to be gained? • Are the plans for inclusion of minorities, both genders and children adequately addressed? • Is the proposed study exempt from human subject review? • No page limits * “Risks” include the possibility of physical, psychological, social injury or safety resulting from research.

34. AHRQ Requirement:Inclusion of Priority Populations • Inclusion of Children? - Rationale? • Inclusion of Elderly? - Rationale? • Inclusion of Rural? - Rationale? • Inclusion of Inner City? - Rationale? • Inclusion of Low Income? - Rationale? • Inclusion of Disabled? - Rationale? • Inclusion of Chronic Care? - Rationale? • Inclusion of End of Life? - Rationale?

35. Review Criteria (Cont.) • For FOAs (RFAs or PAs) , modified or additional criteria may be specified • For other mechanisms, specific criteria apply

36. Peer Review Process • Each application assigned to 3 reviewers for written comments • Streamlined Review may be used • Group discussion of each application, including budget recommendation • Recommended score range • Reviewers privately assign priority score

37. Peer Review Group Actions • Unscored (Streamlined Review) • Scientific Merit Rating (Priority Score)

38. Priority Scores/ Percentiles • Priority scores range from: 100 (Best) to 500 (Worst) [100 x (1.0 – 5.0)] • In Streamlined Review, those in lower half are eliminated (unscored) • Percentiles calculated to normalize scoring behavior across review groups

39. Priority Scores Assigned by Reviewers SCORED (stronger)1.0  1.4 (Outstanding) 1.5  2.0 (Excellent) (25%) 2.0  3.0 (25%) UNSCORED (weaker ) 3.0  4.0 4.0  5.0 (50%)

40. HINTS: PREPARING AN APPLICATION Start early Read and follow instructions Clarify any confusing instructions early Do not assume staff or reviewers “will know what you mean” Include well designed tables and figures Format - consider the reviewers Proofread and check before sending

41. PREPARING AN APPLICATION (cont.) Explicitly state the purpose of the proposed work Refer to the literature thoroughly but thoughtfully Present an organized, lucid write-up Be mindful of the review criteria to be used Get advice from colleagues

42. Art of Persuasion: Assumptions • that the reviewers are knowledgeable and committed to doing a thorough job of evaluating each application • that the reviewers have less time to complete the task than desirable • that good formatting will assist reviewers to remember your organization and the thrust of your arguments

43. Strategies to Strengthen Applications: The Short List Ask scientist with AHRQ support to critically review your application prior to submission – more than once Talk with an Agency (AHRQ, etc.) program officer(s)

44. Strengthening Applications - The Long List: • Significance of problem: Best if transparent yet not insultingly obvious • Scientific Rationale: New &/or original ideas clearly identified • Include a plan for acquiring needed expertise, if needed • Discuss the limitations of the chosen approach • Include abundant detail about the methodology • Scope of work midway on the trivial – grandiose continuum • Describe the participants & their involvement in the study in detail • Include pilot or preliminary data • Describe future research directions

45. Help Reviewers See the Merits • Think like a reviewer – • Learn as much as possible about The System (e.g., figure out the likely review group) • Preempt criticism • Include collaborators who can compensate for your deficiencies • Learn as much as possible about what research projects in your area that have been funded by the NIH/AHRQ/HRSA/CDC/etc. CRISP Database: (http://crisp.oit.nih.gov)

46. http://crisp.oit.nih.gov

48. “In God We Trust….All Others Must Bring Data.” Olivia Bartlett, Ph.D. Chief, Research Programs Review Branch Division of Extramural Activities, NCI

49. BEST WISHES TO YOUR EDUCATION AND PROFESSIONAL ENDEAVORS! ANY QUESTIONS?

50. Human Subjects Protections Overview for Applicants Patrick McNeilly, Ph.D. September 2008