Research Performance Progress Report (RPPR) Implementation by NIH ) February 14, 2013

1. Research Performance Progress Report (RPPR) Implementation by NIH) February 14, 2013 Annie Publow, Director, OSP, Government/NonProfit

2. RPPR: What and Why? TheWhat: “RPPR” (Research Performance Progress Report) A federal initiative that defines common reporting categories for interim performance reports from awardees Implemented by all federal agencies in their own web system The Why: Consistency Standardize types of information requested across all federal agencies that administer research awards Streamline the ability to compare outputs and outcomes of research programs across the government Reduce costs through use of uniform criteria The How: Agency specific web interface eraCommons for NIH, Research.gov for NSF, etc.

3. RPPR Standard Reporting Categories: NIH will require completion of all of the following report categories: Cover Page Accomplishments Products Participants Impact Changes Special Reporting Requirements (as applicable) Budget (non-SNAP awards only) The next series of slides describe what will be included for that category

4. RPPR Standard Cover Page Elements The standard cover page data elements, as well as mandatory and optional components comprise the complete research performance progress report format. If an agency has an electronic reporting system that can identify the award and recipient, it is not required to collect the standard institutional information included in the cover page data elements.

5. RPPR Standard Reporting Categories:Accomplishments Accomplishments is a mandatory category for all federal agencies Category answers the questions: What was done? What was learned?  What are the major goals and objectives of the project?  What was accomplished under these goals?  What opportunities for training and professional development has the project provided?  How have the results been disseminated to communities of interest?  What do you plan to do during the next reporting period to accomplish the goals and objectives? Allows the agency to assess whether satisfactory progress has been made during the reporting period

6. RPPR Standard Reporting Categories:Products Products is an optional category that will be used by some agencies for some projects including NIH Category answers the question: What has the project produced?  Publications, conference papers, and presentations;  Website(s) or other Internet site(s);  Technologies or techniques;  Inventions, patent applications, and/or licenses; and  Other products, such as data or databases, physical collections, audio or video products, software or NetWare, models, educational aids or curricula, instruments, or equipment Allows the agency to evaluate what the publications demonstrate about the excellence and significance of the research and the efficacy with which the results are being communicated to colleagues, potential users, and the public. Products beyond publications allow an agency to assess and report to Congress, communities of interest, and the public.

7. RPPR Standard Reporting Categories:Participants & Other Collaborating Organizations Participants & Other Collaborating Organizations is an optional category that will be used by some agencies for some projects including NIH Category answers the question: Who has been involved?  What individuals have worked on the project?  What other organizations have been involved as partners?  Have other collaborators or contacts been involved? Allows an agency to know who has worked on the project to gauge and report performance in promoting partnerships and collaborations.

8. RPPR Standard Reporting Categories:Impact Impact is an optional category (that will be used by some agencies for some projects including NIH) Category answers the questions: What is the impact of the project? How has it contributed? Describe distinctive contributions, major accomplishments, innovations, successes, or any change in practice or behavior that has come about as a result of the project relative to:  the development of the principal discipline(s) of the project;  other disciplines;  the development of human resources;  physical, institutional, and information resources that form infrastructure;  technology transfer (include transfer of results to entities in government or industry, adoption of new practices, or instances where research has led to the initiation of a startup company); or  society beyond science and technology. Allows an agency to know who has worked on the project to gauge and report performance in promoting partnerships and collaborations. Allows agency to provide the taxpaying public and its representatives with an assessment of how the investments they make benefit the nation. Over time, this base of knowledge, techniques, people, and infrastructure is drawn upon repeatedly for application to commercial technology and the economy, to health and safety, to cost-efficient environmental protection, to the solution of social problems, to numerous other aspects of the public welfare, and to other fields of endeavor. Through this reporting format, and especially this section, recipients provide that assessment and make the case for Federal funding of research and education.

9. RPPR Standard Reporting Categories:Changes/Problems Changes/Problems is an optional category (that will be used by some agencies for some projects including NIH) Category answers the question: How is it going?  Changes in approach and reasons for change.  Actual or anticipated problems or delays and actions or plans to resolve them.  Changes that have a significant impact on expenditures.  Significant changes in use or care of animals, human subjects, and/or biohazards. Allows for agency understanding and involvement during course of project. Reminder grantee is required to obtain prior written approval from the awarding agency grants official whenever there are significant changes in the project or its direction.

10. RPPR Standard Reporting Categories:Special Reporting Requirements Special Reporting Requirementsis an optional category (that will be used by some agencies for some projects including NIH) Category answers agency- and/or project-specific questions  Will vary….. Allows the agency to assess progress toward or outcome of defined concerns

11. RPPR Standard Reporting Categories:Budgetary Information Budgetary Information is an optional category (that will be used by some agencies for some non-SNAP projects) Category will be used to collect budgetary data from the recipient organization to be used in conducting periodic administrative/budgetary reviews. One of the standard budgetary formats, as identified by the agency, will be used to submit this information. Note: These budgetary forms are identical to those included in the SF 424 R&R forms family, as implemented by Grants.gov. As these forms are updated in Grants.gov, these budget forms will change accordingly.

12. RPPR Standard Reporting Categories:Demographic Information for Significant Contributors Demographic Information for Significant Contributors is an optional category (that will be used by some agencies for some projects) Agencies may require that recipients provide demographic data about significant contributors for a variety of purposes, including:  Gauge whether our programs and other opportunities are fairly reaching and benefiting everyone regardless of demographic category;  Ensure that those in under-represented groups have the same knowledge of and access to programs, meetings, vacancies, and other research and educational opportunities as everyone else;  Gauge and report performance in promoting partnerships and collaborations;  Assess involvement of international investigators or students in work we support;  Track the evolution of changing science, technology, engineering and mathematics (STEM) fields at different points in the pipeline (e.g., medicine and law demographics have recently changed dramatically);  Raise investigator and agency staff awareness of the involvement of under-represented groups in research;  Encourage the development of creative approaches for tapping into the full spectrum of talent of the STEM workforce;  Respond to external requests for data of this nature from a variety of sources, including NAS, Congress, etc.; and  Respond to legislatively-required analysis of workforce dynamics. Legislation requires at least one agency to routinely estimate scientific workforce needs. This analysis is accomplished through reviewing demographic data submitted for the existing workforce.

13. RPPR Implementation Plans by Agency Every agency will direct recipients to report on the one mandatory component →Accomplishments An agency may direct a recipient to report on optional components, as appropriate. Within a component, agencies may direct recipients to complete only specific questions, as not all questions within a given component may be relevant to all agencies.

14. RPPR Implementation Plans by Agency Agencies may develop an agency- or program-specific component, if necessary, to meet programmatic requirements, although agencies should minimize the degree to which they supplement the standard components. Such agency- or program- specific requirements will require review and clearance by OMB. Agencies also may use other OMB approved reporting formats, such as the Performance Progress Report (PPR), if those formats are better suited to the agency’s reporting requirements, for example, for research centers/institutes, clinical trials, or fellowship/training awards or in connection to reporting on program performance.

15. NIH Requires Use of RPPR for All SNAP and Fellowship Progress Reports, and Expands RPPR Functionality Research Performance Progress Report (RPPR) Notice Number: NOT-OD-13-035 Release Date: February 6, 2013 http://grants.nih.gov/grants/guide/notice-files/NOT-OD-13-035.html

16. NIH Implementation of RPPR All NIH grantees will have the option to use the RPPR for Streamlined Non-competing Award Process (SNAP) and Fellowship progress reports beginning 10/19/2012. Grantees may continue to use eSNAP or paper submissions as appropriate for reports due through April 2013. Use of the RPPR will be mandatory effective with SNAP-eligible and Fellowship progress reports beginning May 2013 (budget start date of July 1, 2013 or later.) NIH also anticipates piloting the RPPR for non-SNAP awards during calendar year 2013.

17. NIH Implementation of RPPR: SNAP-eligible Non-Fellowship, SNAP-eligible Awards: PI has option of submitting progress report in SNAP or RPPR format through April 2013 after which RPPR is mandatory. Until May, PI will be prompted to choose either SNAP or RPPR format when logging in to process report. Once started, the only way to change the progress report format is with the assistance of the eRA Help Desk at Commons Support (1-866-504-9552 or commons@od.nih.gov). Regardless of format, progress reports are due the 15th of the month preceding the month in which the budget period ends. SNAP-eligible progress reports will continue to be prepared by the PI in eraCommons and released to OSP for review and submission to NIH. Research Performance Progress Reports are also prepared by the PI in eraCommons and released to OSP for review and submission to NIH. Login to eraCommons is using the NIH Username and password Access is by Role

18. NIH Implementation of RPPR: Fellowships Fellowship Awards: Fellowship progress reports are due on the 1st of the month, two months before the beginning date of the next budget period. Fellow has option of submitting progress report in paper or RPPR electronic format through April 2013 after which RPPR is mandatory. Paper progress reports are prepared by the Fellow and forwarded to OSP two weeks prior to sponsor deadline for review and Authorized Official signature. Signed documents are returned to the Fellow for submission to NIH. Electronic Research Performance Progress Reports (RPPR) are prepared by the Fellow in eraCommons and released to OSP two weeks prior to sponsor deadline for review and submission to NIH in eraCommons. Login to eraCommons is using the NIH Username and password Access is by Role

19. SNAP vs. RPPR • The RPPR and eSNAP have a number of differences: • RPPR has separate screens for each of the following reporting components: Cover Page, Accomplishments, Products, Participants, Impact, Changes, Special [agency specific] Reporting Requirements, and Budget [applicable only for non-SNAP awards] • When implemented for non-SNAP awards the Budget component will be a SF424(R&R) Budget. • The format of the report will be new. • Users will answer questions by using a checkbox, entering text or uploading a PDF, or selecting “Nothing to Report.” • New information to be provided by grantees through the RPPR includes: • Foreign component information • Dollars spent in foreign country(ies) [through first-tier subawards] • Organizational affiliation of personnel at foreign sites • Effort on All Personnel report will be rounded to nearest whole person month. • Other features of the RPPR include: • Specific location to report on competitive revisions/administrative supplements associated with the award. • Public Access compliance status will be displayed • Other support will only be required if there has been a change • Notice of Award link • Streamlined reporting of ClinicalTrials.gov information The RPPR and eSNAP modules have a number of similarities: The substance of the report is similar; the grantee will be asked to describe progress, study results, the significance of the findings, and any significant changes. Where possible, information is pre-populated from NIH systems for the grantee, including PD/PI information, grant number, project title and period, performance sites, and personnel. Publications in PD/PI’s MyNCBI account will be displayed for easy association with the progress report. No need to submit a detailed budget. Information required by NIH policies will continue to be requested from grantees. For example, the RPPR will address policies covering such areas as human subjects education, inclusion enrollment reporting, and use of human embryonic stem cells.

20. NIH Implementation of RPPR: PRAM Progress Report Additional Materials (PRAM): PRAM is a feature of the RPPR and allows the grantee to electronically enter, review and submit information in response to follow up requests from NIH for additional materials. PRAM will be fully functional by April 18, 2013 at which time it is the recommended means for responding to grants management or program official requests for additional information in connection with a submitted progress report. All investigators should familiarize themselves with the NIH Public Access Policy to avoid PRAM requests and/or delays to funding because of publications-related reporting requirements: http://publicaccess.nih.gov/

21. Sample Email to PI from Program Officer when Publications reporting is delinquent Can delay issuance of continuing award These requests moving to “PRAM”

22. NIH Public Access Policy and Compliance Overview (http://publicaccess.nih.gov/) The NIH Public Access Policy ensures that the public has access to the published results of NIH funded research. It requires scientists to submit final peer-reviewed journal manuscripts that arise from NIH funds to the digital archive PubMed Centralupon acceptance for publication. To help advance science and improve human health, the Policy requires that these papers are accessible to the public on PubMed Central no later than 12 months after publication. The Difference Between a PMCID and a PMID The PubMed Central reference number (PMCID) is different from the PubMed reference number (PMID). PubMed Central is an index of full-text papers, while PubMed is an index of abstracts. The PMCID links to full-text papers in PubMed Central, while the PMID links to abstracts in PubMed. PMIDs have nothing to do with the NIH Public Access Policy. Progress Report Requirements Your progress report must include the PMC reference number (PMCID) when citing applicable papers that you author or that arise from your NIH-funded research. Here is a clear explanation of the rules: Link: http://publicaccess.nih.gov/citation_methods.htm The bottom line is that you must have a PMCID number (not a PMID number) for any publication that was published more than 3 months ago. For new pubs or "in press" pubs, there are two options, see link above for explanation. A PMCID is the only way to demonstrate compliance.

23. Helpful links for understanding RPPR (and identification of sources) Overview on NSF website: http://www.nsf.gov/bfa/dias/policy/rppr/index.jsp Overview on NIH website: http://grants.nih.gov/grants/rppr/ RPPR Format as approved by OMB/OSTP: http://www.nsf.gov/bfa/dias/policy/rppr/format_ombostp.pdf RPPR resulted from an initiative of the Research Business Models (RBM) Subcommittee of the Committee on Science (CoS), a committee of the National Science and Technology Council (NSTC)