1 / 82

AFP and Telework Program Review

AFP and Telework Program Review. Jeremy Buzzell and Robert Groenendaal. Requirements for Program Review. The Handbook for the Discretionary Grant Process Section 5.3.2 Monitoring Policy

anjolie-beard
Download Presentation

AFP and Telework Program Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AFP and Telework Program Review Jeremy Buzzell and Robert Groenendaal

  2. Requirements for Program Review The Handbook for the Discretionary Grant Process Section 5.3.2 Monitoring Policy It is ED’s policy to monitor active discretionary grants with a focus on technical assistance, continuous improvement, and attaining promised results. Monitoring a grantee shall continue for as long as ED retains a residual financial interest in the project, whether or not ED is providing active grant support.

  3. Requirements for Program Review Section 5.3.2 Monitoring Policy ED staff is to monitor each grantee to the extent appropriate so as to achieve expected results under approved performance measures, while assuring compliance with grant requirements. Existing requirements in EDGAR extend equally to all grantees and their partners.

  4. Requirements for Program Review Section 5.3.3 Monitoring Purpose Systematic and regular monitoring by ED staff of a grantee’s activities measures the project quality and progress, including strengths and weaknesses.

  5. Requirements for Program Review 1. A grantee’s project must: a) Conform to the grantee’s approved application and any approved revisions, as well as the effectiveness and quality of the project (Program Management); b) Progress against previously established performance measures (Performance Measurement); c) Adhere to laws, regulations, conditions of the grant, certifications and assurances (Compliance); and d) Manage Federal funds according to Federal cash management requirements, including expenditures of funds for authorized purposes (Fiscal Accountability).

  6. Requirements for Program Review 2. Regular monitoring enables ED staff to provide customized technical assistance, appropriate feedback, and follow-up to help grantees to: • Improve areas of need • Identify project strengths • Recognize significant achievements

  7. Development of Program Review • Open forum in Denver • Review of existing protocols • ED training in auditing • Developed proposals • NATTAP Blog • Teleconference of directors in September 2007 • Refined proposals • Internal review and approval of proposals

  8. What we learned from the process . . . • Proof of compliance should not be based on data alone. • Every grantee should be assessed the same way but can show compliance in its own way. • Grantees should not be compared to each other. • Compliance should not be black or white. • Look for the positive as well as the negative.

  9. The result . . . • Program Reviews allow for a holistic explanation of the program (through documents, data, and discussion). • This explanation is done for peer reviewers and RSA. • The peer reviewers and RSA answer a series of questions about the program to determine the extent of compliance.

  10. The effect . . . Pros • Context is considered • Focuses on what you can control • Appropriate to size and funding of program Cons • Time-intensive (for all) • Highly subjective • Learning curve for RSA and grantees

  11. Why this approach? • On-site reviews are not feasible. • RSA staff are limited and do not come from the field. • Data is not reliable. • Lack of basis for self-assessment. • Program continues to evolve. • One size fits all.

  12. Next Steps with Program Review • Final drafts cleared by RSA. • Final drafts reviewed by Risk Management. • Official materials shared with AFP/Telework Programs. • Choose and notify first round of programs. • AFP/Telework Programs submit FY 2008 data. • Train third-party (peer) reviewers. • First peer review in January 2009.

  13. Role of Lead Agencies and Community Based Organizations in Program Reviews • The LEAD AGENCY is being reviewed. • Participation from BOTH the Lead Agency and CBO is expected. • Participation of all personnel being paid using grant funds is required (see exception in the next bullet). • Participation of the Certifying Representative is encouraged but not required.

  14. How are AFP/Telework Programs selected for review? Random selection of an equal number of grantees in two categories: (1) Title III AFP/Telework grantees claimed by a Statewide AT Program as a state financing activity; OR (2) Title III AFP/Telework grantees not claimed by a Statewide AT Program as a state financing activity.

  15. How are AFP/Telework Programs selected for review? Exceptions to random selection: • Recent major change in the AFP/Telework Program or State Plan for AT • Recent major disaster • RSA is concerned

  16. How are AFP/Telework Programs selected for review? • 6 grantees per cycle (3 x 3) • A “cycle” is a fiscal year (Oct. 1 to Sept. 30) • 9 weeks per grantee (some program reviews may overlap) • First year of program reviews: • Jan. 1 to Sept. 30 • Only 3-4 reviews

  17. What happens when an AFP/Telework Program is selected? • RSA notifies grantee in spring prior to next cycle. • RSA and grantee negotiate 9-week window within next cycle. • RSA recruits third-party (peer) reviewers. • Grantee begins preparing for review.

  18. How does a grantee prepare for the review? A review assesses two “Core Components” of grant implementation, each in a slightly different way. • Core Component 1 – Program Management • Core Component 2 – Program Performance

  19. Elements of Program Management Fiscal Management Contract Oversight Personnel Management Elements of Program Performance Emphasis and Expansion of Consumer Choice and Control Timely and Equitable Access to Financial Assistance Capacity to Continue on a Permanent Basis Core Components of Program Review

  20. Program Management The review of Program Management verifies that a grant is managed efficiently and appropriately in accordance with title III, the NFP, EDGAR, and OMB Circulars, as applicable. This includes: • Fiscal Management • Contract Oversight • Personnel Management

  21. Fiscal Management EDGAR requirements: • Fiscal control and accounting procedures to insure proper disbursement of and accounting for Federal funds; controls and procedures adequate to establish that such funds have not been used in violation of applicable statutes. • Records that fully show the amount of funds under the grant, how the grantee uses the funds, the total cost of the project, the share of the cost provided by other sources, and other records to facilitate an effective audit. • Audits in accordance with OMB A-133.

  22. Fiscal Management Title III or NFP requirements: • Assurance that the grantee will provide the non-federal share of the cost of the program in cash, from state, local, or private sources. • Assurance that the grant funds will be used to supplement and not supplant other federal, state, and local public funds. • Assurance that the grant funds will be placed in a permanent separate account and identified and accounted for separately from any other fund. • Assurance that the funds are administered, expended, and invested in accordance with title III or the NFP.

  23. How to prepare for a review of Fiscal Management Submit to RSA: • Expenditure report. • Proof of title III or NFP fiscal management assurances. • Approved indirect cost rate agreement/cost allocation plan or questionnaire provided by RSA (applies to both Lead Agencies and Community Based Organizations.) • Copies of most recent audit reports or questionnaire provided by RSA (applies to both Lead Agencies and CBOs). (In addition, RSA will look at ED 524Bs and GAPS records.)

  24. Expenditure Report • 1 for Lead Agency to administer the grant, 1 for CBO to implement the program. • From the most recent federal fiscal year. • There is no “format”, but RSA must be able to understand the reports. • Show use of all federal and match funds as well as income from interest, investments, loan repayments, and other sources. • Show use of funds used for administration of the program separately from the amount used for loans and other program purposes. • Show the amount of funds used for indirect costs.

  25. Expenditure Report What RSA is looking for: • Existence, quality, and legitimacy of report • Permanent separate account • Compliance with indirect cost limits

  26. Proof of Fiscal Management Assurances What RSA is looking for: • The sources, amounts, and appropriateness of match provided for all grants for the AFP/Telework Program. • The Lead Agency’s and CBO’s internal controls that ensure all principal and interest from the permanent separate account (a) support only AFP/Telework activities and (b) supplement and do not supplant other federal, state, and local public funds expended to provide financial loans.

  27. Indirect Cost Rates What RSA is looking for: • Existence of agreement • Appropriate kind of agreement • Handling of indirect costs between Lead Agency and CBO • Amount of indirect will be seen in expenditure reports

  28. Audits What RSA is looking for: • An audit was done (if required) • Findings of the audit • Resolution of findings

  29. ED 524B Forms and GAPS Records What RSA is looking for: • Completeness and accuracy of financial reports

  30. Contract Oversight EDGAR requires grantees to: • Directly administer or supervise the administration of each project. • Have procedures for providing TA, evaluation and for performing other administrative responsibilities necessary to ensure compliance. • Monitor activities to assure compliance with applicable Federal requirements and that performance goals are being achieved.

  31. Contract Oversight Grantees are required to enter into a contract with a CBO or group of CBOs that has individuals with disabilities involved in the organizational decision-making to administer the AFP/Telework Program. The contract must: • Include a provision requiring funds be administered consistent with title III or the NFP. • Include any provision the Secretary requires concerning oversight and evaluation of federal financial interests. • Require the CBO(s) enter into a contract with commercial lending institutions or organizations or state financing agencies.

  32. How to prepare for a review of Contract Oversight Submit to RSA: • A dated copy of the grantee’s most recent contract with the CBO

  33. How to prepare for a review of Contract Oversight What RSA is looking for: • Appropriate oversight to protect Federal interests and ensure implementation • Relevant to title III or the NFP

  34. Personnel Management OMB Circulars: • Compensation to personnel must be reasonable. • Charges to Federal awards for salaries and wages are based on documented payrolls. • Charges for salaries and wages of employees who work solely on a single Federal award are supported by periodic certifications. • Charges for salaries and wages of employees who work on multiple activities are supported by personnel activity reports.

  35. How to prepare for a review of Personnel Management Submit to RSA either: • Position descriptions of record for those being paid with grant funds and personnel activity reports for them; OR • Synopsis for key personnel describing his or her role and responsibilities associated with the grant.

  36. Personnel Management What RSA is looking for: • Appropriate number of personnel • Appropriate time commitments of personnel • Appropriate duties for personnel

  37. How to prepare for a review of Program Management • Submit all the previously described documents. • RSA reviews the documents. • RSA discusses the documents via conference call (the grantee is responsible for bringing together the appropriate parties). • RSA uses this information to answer questions about program management. • Yes/no with explanation.

  38. Questions about Program Management 1. Has the grantee obtained its full matching funds from appropriate sources? If not, does the grantee have a plan for obtaining the remaining match funds? 2. Does the grantee account for grant funds accurately and completely in accordance with EDGAR? 3. Does the grantee use grant funds only for purposes under, and within the limitations of, title III or the NFP, as applicable? 4. Does the grantee comply with indirect cost requirements? 5. Does the grantee comply with audit requirements?

  39. Questions about Program Management 6. Does the grantee exercise appropriate fiscal and performance oversight of the AFP/Telework Program through its contract with the CBO? If not, does the Lead Agency have a plan for improving its oversight? 7. Is the contract structured to meet the requirements and intent of title III of the NFP, as applicable? If note does the Lead Agency have a plan for changing its contract to make it more consistent with title III or the NFP, as applicable?

  40. Questions about Program Management  8. Does the AFP/Telework Program have an appropriate number of staff to implement the grant? If not, does the Lead Agency have a plan for changing its contract to make it more consistent with title III or the NFP, as applicable?  9. Do the Lead Agency and CBO distribute the time of their personnel appropriately to implement the grant based on their responsibilities? If not, does the grantee have a plan to distribute time more appropriately? 10. Do the Lead Agency and CBO use grant funds to support only personnel with responsibilities germane to the grant?

  41. Assessing Compliance of Program Management There is no overall rating of compliance. For each element of Program Management, a grantee will receive 1 of 3 ratings: • Compliant • In Need of Improvement • Fails to Substantially Comply

  42. Assessing Compliance of Program Management The rating is based on how RSA answers the questions: • Initial question: Does the AFP/Telework Program have an appropriate number of staff to implement the grant? • Subquestion: If not, does the grantee have a plan for adjusting the number of staff?

  43. Assessing Compliance of Program Management • If the answer to the initial question is “Yes” – the grantee is rated as compliant. • If the answer to the initial question is “No” – the rating depends on the answer to the subquestion. • If the answer to the subquestion is “Yes” – the grantee is rated as in need of improvement. •  If the answer to the subquestion is “No” – the grantee is rated as failing to substantially comply.

  44. Assessing Compliance of Program Management Questions for which “needs improvement” doesn’t apply: • Does the grantee account for grant funds accurately and completely in accordance with EDGAR? • Does the grantee use grant funds only for purposes under, and within the limitations of, title III or the NFP, as applicable? • Does the grantee comply with indirect cost requirements? • Does the grantee comply with audit requirements? • Does the grantee use grant funds to support only personnel with responsibilities germane to the grant?

  45. Assessing Compliance of Program Management The onus is on the grantee to present evidence/plan sufficient to get to a “yes” answer. • RSA will ask questions or provide examples of sufficient evidence, if needed. • Grantee likely will know their status before the conversation is over.

  46. Program Performance The review of Program Performance assesses whether the AFP/Telework Program is implementing its grant in accordance with title III or the NFP, as applicable. This includes: • Extent to which the program emphasizes and expands consumer choice and control. • Extent to which program provides timely and equitable access to financial assistance. • Extent to which the program can continue on a permanent basis.

  47. Third-Party Reviewers Peer reviewers participate in the review of Program Performance: • 3 current or former program directors (or position with similar duties). • At least 1 with prior review experience on each team. • Trained, paid volunteers from a standing pool. • No reviewers from NATTAP or NISAT. • Participation is not anonymous, but conclusions are not attributed to individual reviewers. Peer reviewers learn about the program through review of documents, looking at data, and discussion on teleconferences/webinars.

  48. How to prepare for a review of Program Performance Submit to RSA: • Narrative explaining how consumer choice and control is emphasized and expanded. • Copies of administrative policies and procedures manuals. • Copies of lender agreement(s), investment plan, if applicable, and narrative or plan for sustainability. • Reviewers already will have AFP/Telework Program data. There is no prescribed format for these but must be accessible.

  49. How to prepare for a review of Program Performance • Develop teleconference/webinar presentations that serve as a virtual on-site visit to your program. • Length, format, and content of presentation are up to the program. • The “presenter” during a teleconference or webinar is up to the program. • Need to be accessible.

  50. Emphasis and Expansion of Consumer Choice and Control Title III or NFP requirements: • Assurance that, and information describing the manner in which, the grantee will expand and emphasize consumer choice and control.

More Related