Program Review and Best Practices: Who, What, When, Where, Why, How?

Program Review and Best Practices: Who, What, When, Where, Why, How? Jeremy Buzzell, Program SpecialistDenver, CO. May 23, 2007

Program Review: Why? The Handbook for the Discretionary Grant Process (February 2006 Edition) Section 5.3.2 Monitoring Policy It is ED’s policy to monitor active discretionary grants with a focus on technical assistance, continuous improvement, and attaining promised results. Monitoring a grantee shall continue for as long as ED retains a residual financial interest in the project, whether or not ED is providing active grant support.

Program Review: Why? • Section 5.3.2 Monitoring Policy • ED staff is to monitor each grantee to the extent appropriate so as to achieve expected results under approved performance measures, while assuring compliance with grant requirements. Existing requirements in EDGAR extend equally to all grantees and their partners.

Program Review: Why? • Section 5.3.3 Monitoring Purpose • Systematic and regular monitoring by ED staff of a grantee’s activities measures the project quality and progress, including strengths and weaknesses.

Program Review: Why? • A grantee’s project must: • a) Conform to the grantee’s approved application and any approved revisions, as well as the effectiveness and quality of the project (Program Management); • b) Progress against previously established performance measures (Performance Measurement); • c) Adhere to laws, regulations, conditions of the grant, certifications and assurances (Compliance); • d) Manage Federal funds according to Federal cash management requirements, including expenditures of funds for authorized purposes (Fiscal Accountability).

EDGAR Section 80.43 Enforcement • Remedies for noncompliance • (1) Temporarily withhold cash payments pending correction of the deficiency by the grantee or subgrantee or more more severe enforcement action by the awarding agency. • (2) Disallow (that is, deny both use of funds and matching credit for) all or part of the cost of the activity or action not in compliance. • (3) Wholly or partly suspend or terminate the current award for the grantee’s or subgrantee’s program. • (4) Withhold further awards for the program, or • (5) Take other remedies that may be legally available.

Program Review: How? • The program review is a one-size fits all protocol.

Program Review: How? • Proposed Process for Developing Program Review Protocol: • Input from NATTAP Annual Conference in Denver • Review of existing OSERS protocols • Internal development of ideas based on (1) and (2) • Share and seek feedback on ideas • Refine ideas based on (4) – develop written draft • Share and seek feedback on draft • Refine draft based on (6) – develop final • Internal review of final • Roll-out

Program Review: What? • “If a grantee or subgrantee materially fails to comply with any term of an award, whether stated in a Federal statute or regulation, an assurance, in a State plan or application, a notice of award, or elsewhere . . .” • Applicable assurances: • Match • Permanence of the Program • Consumer Choice and Control • Permanent Separate Account • Use and Control of Funds • Contract with a Community-Based Organization • Administrative Policies and Procedures • Data Collection

Program Review: How? • What does failure to comply mean? Can it be measured quantitatively using data?

Program Review: How? • What does failure to comply mean? If it cannot be measured quantitatively, what can/must be measured qualitatively? How is this qualitative measurement done?

Program Review: When? Program reviews to begin in FY 2008 (3rd year of plan) Limitation: No more than 6 a year given staff time and budget Process must be selective On what basis is a state chosen for review?

Program Review: When? On what basis is a state chosen for review? Performance? Random? Alphabetical? Population? Region? Grant size? Program maturity? Other dimension or combination of dimensions?

Program Review: Where? Must a program review be done on-site?

Program Review: Where? • What does on-site mean? • What is “seen” on-site? • What can be done off-site?

Program Review: Who? Who is involved in the review?

Program Review: Who? Who is involved in the review? • RSA staff • External experts? • Internal staff? • Consumers/stakeholders? • Subcontractors?

Program Review: Then What? • Program reviews generally conclude with a written report. • These reports are public information. • What should we be able to say about a program when the review is complete?

Focus on Best Practices: Why? • Grantees have requested that NATTAP’s TA focus on disseminating information about best practices. • Remedies for non-compliance should include best practices.

What are “Best Practices?” Best Practice is a superior method or innovative practice that contributes to the improved performance of an organization, usually recognized as "best" by other peer organizations. It implies accumulating and applying knowledge about what is working and not working in different situations and contexts, including lessons learned and the continuing process of learning, feedback, reflection and analysis (what works, how and why). -- Visitask project management resource center

Program Review and Best Practices: Who, What, When, Where, Why, How?