State of Florida Agency for Workforce Innovation Office of Early Learning

1. State of FloridaAgency for Workforce InnovationOffice of Early Learning School Readiness Program Accountability

2. Background • The School Readiness Act established the Partnership for School Readiness in 1999 • Prior to 1999 the Subsidized Child Care Program was run through 25 central agencies throughout the state • The Partnership contracted with 57 coalitions to provide school readiness services • Most of these coalitions contracted with the “historical central agencies” • The Office of Early Learning was established within the Agency for Workforce Innovation on January 2, 2005

3. Background • The Office of Early Learning administers the School Readiness program (SR) through: • Grant Agreements with 31 Early Learning Coalitions • Statewide Contract with Redlands Christian Migrant Association (RCMA ) • Statewide Contract for Home Instruction for Parents of Preschool Youngsters (HIPPY) • Annual budget of $653 million (2008-09) • SR served approximately 247,000 children (2008-09) • The Office of Early Learning also administers the Voluntary Prekindergarten Education program through Early Learning Coalitions

4. Background SR System Basics • All coalitions sign a grant agreement annually • System delivery varies with coalitions: • Direct service delivery • Contract out service delivery • Combination of both • All child care providers sign an agreement with the coalition or the contracted service provider • Providers must meet minimum health and safety guidelines and pass a criminal background check

5. Background Prior to establishment of the Office of Early Learning • Coalitions received one financial monitoring visit each year • Focused on review of expenditures • Coalitions required to conduct attendance monitoring The Office of Early Learning was established in 2005 • Financial Monitoring Systems Assurances Section was established within the Agency and began a comprehensive financial monitoring program • Implemented triennial performance review including limited eligibility monitoring

6. Issue and Reaction In 2006-2007, The Office of Early Learning participated in the 2nd USDHHS Error Rate Pilot Study • Improper Payment Error rate was 18% As a Result • Eligibility and Fraud Prevention Unit was established • Developed SR Desk Reference Guide • eligibility factors • proper documentation techniques • Conducted statewide webinars on accurate eligibility and proper documentation techniques Most Recent Improper Payment Error Rate = 2.7%

7. Issue and Resolution Results of reaction to Error Rate Pilot • Many coalitions were concerned over excessive monitoring visits and duplication • Coalitions had the following monitoring reviews: • Triennial Performance Monitoring • Quarterly Invoice Desk Review • Financial Monitoring • Eligibility Monitoring The Agency implemented a monitoring streamlining project to: • Eliminate duplication • Develop centralized document library • Develop statewide eligibility programs • Combine Performance and Eligibility Monitoring

8. Issue and Resolution Streamlining Project Successful • Duplication Eliminated • Centralized Document Repository • Statewide Standard Eligibility Monitoring Programs • Combined Performance and Eligibility Monitoring Why are standard monitoring programs necessary? • 14 coalitions deliver all services directly • 17 coalitions contract services to contracted service providers Office of Early Learning can monitor coalitions “in-house” and validate results of coalition subrecipient monitoring and ensure the same requirements are covered and monitored using the methodology

9. Current Monitoring Practices • Financial • Complete and Internal Control Assessment each year • Results of assessments help to shape the on-site review • Annual On-site Financial Monitoring • Quarterly desk review of invoice • Sample items selected are not duplicative of samples selected during annual review • Programmatic - Annual Review (on-site or desk review) • Data Quality Reports

10. Current Monitoring Practices • Financial Monitoring Includes: • Cash Management • Prepaid Program Items (Participant Support Costs) • Cost Allocation and Disbursement Testing • Purchasing Activities • Contract Activities • Subrecipient Monitoring • ARRA-Related Reporting Requirements • Internal Controls • General Conditions

11. Internal Control Survey Sample

12. 09-10 AWI Financial Monitoring ToolExecutive Summary

13. Current Monitoring Practices Programmatic Monitoring Includes • Contracts • Educational Service Delivery • Board Governance • Child Care Resource and Referral • Operational Program Management • Coalition Plan Implementation • ARRA Funding • Data Security • Eligibility • File Review • Payment/Attendance Accuracy • Local Procedures and Practices

14. Current Monitoring Practices Programmatic: Data Security • Coalitions are required to develop data security protocols internally and to ensure their subrecipients have the same protocols in place • During the annual review the following protocols are reviewed: • Security Training and Awareness, • Contingency Planning, • Access Control, • Identification and Authentication, • Personnel Security, • Mobile Computing, • Remote Access, • Database Security, • Media Management, • Password Management, • Data Integrity, and • Records Confidentiality Compliance

15. Example of Data Security Criteria Data Security Integrity Performance Criteria (DSI-A) - EFS Data and Reporting Systems Accuracy The law requires each coalition to use the Enhanced Field System (EFS), the “single statewide information system” established by the Agency under s. 411.01(5)(c)1.e., F.S. The coalition must use the EFS database to establish eligibility for child care funding. EFS data must accurately reflect the most current household circumstances that the applicant presented to the coalition. EFS data is the basis for reimbursement and federal and state reporting. The review team compared # (SR and VPK) child eligibility files against the EFS database records for accuracy of entry, timeliness of processing actions, case history note narrations, and data security. In addition, the team reviewed the files for correct parent fees, eligibility dates, family income, household size, and correct demographic data. The team reviewed EFS data to ensure that the data matched the application.

16. Example of Data Security Protocol

17. Current Monitoring Practices Programmatic: Eligibility • Standard Eligibility Review Programs finalized in 2009-2010 • Programs include criteria that the Office of Early Learning will review on all coalitions delivering services in-house • Coalitions are required to monitor their subrecipients using the criteria identified in the program • Grant Agreement Requirement • 2009-2010 was the implementation/transition year • Standard monitoring tools and a report template has also been created

18. Example of SR Criteria SR Eligibility Criteria (SR-M) - Family Unit Income A SR family file must include a completed SR Income Worksheet (SR-100) to determine a child’s eligibility and establish the applicable parent fee based on the sliding fee scale included in the coalition’s approved SR plan. If the family counts the individual as a family member, the eligibility specialist should include any income the individual may receive as part of the total family income. If the family lives with an aunt, uncle, or grandparent or other adult whom the family does not support, the specialist should not count the individual or the individual’s income in the family size or income. If a parent/guardian lives with a friend who is not a biological or legal parent/guardian of a minor child that is eligible for care, and the friend does not contribute money or consider themselves part of the household, then the eligibility specialist should not include the friend in the household. Agency rules provide guidelines for verification of employment and income (Rule 60BB-4.208, F.A.C.). For each client, does the coalition have the following information on file: A completed SR Income Worksheet for Eligibility and Parent Copayment(SR-100)? (45 CFR 98.20(2); Rules 60BB-4.100 and 4.208, F.A.C.)YES NO Earned and unearned family income fully supported by the appropriate documentation?(45 CFR 98.20(2); Rule 60BB-4.100 and 4.208, F.A.C.) YES NO Earned and unearned family income correctly calculated on the SR Income Worksheet for Eligibility and Parent Copayment (SR-100)?(45 CFR 98.20(2); Rule 60BB-4.100 and 4.208, F.A.C.)YES NO

19. Current Monitoring Practices Programmatic: Review process • Notification letter • Entrance conference • Review procedures • Exit conference • Draft copy to Executive Director (ED) for courtesy review • Report published and provided to ED and Board Chair • Corrective Action Plan • Technical Assistance throughout the review

20. Current Monitoring Practices Data Quality Reports • Purpose • Data irregularities • Payment rate issues • Data accuracy issues • Standardization of codes • Types • Exception Reports within the system • Reports run at the state level

21. Current Monitoring Practices Examples of Data Quality Reports • Exception Reports • Duplicate Parent/Child/Provider – Shows parents, children, and providers with the same name (but different unique identifiers) • Enrolled but Not Paid – Children enrolled at a provider but without payments. • Family Size and Income Report – Shows detailed family size and income information by parent • Statewide reports • Gold Seal (identifies providers with a Gold Seal payment which have a missing certificate or error with certificate end date) • Purpose for Care Report: If employment listed but no earned income listed, and purpose for care is education/training-no school identified

22. Fraud Referral Database The Office of Early Learning teamed with the Agency’s Inspector General’s Office and the Florida Department of Law Enforcement (FDLE) to develop an automated fraud referral database The Database provides Centralized fraud referrals An automated referral process Monthly status updates

23. Fraud Referral Database How does it work? Fraud Referrals can be made on parents, providers, and/or employees Coalitions input the referrals Office of Early Learning staff reviews referrals before forwarding to FDLE If a fraud referral is received by FDLE for childcare, FDLE checks other programs for violations as well, including TANF and Food Stamps

24. Fraud Referral Database Results Standardized the referral process Trained coalitions on what constitutes fraud Restitution is now standardized and tracked Since 1999, 1,329 fraud referrals were processed In the last 3 years, 878 fraud referrals have been processed

25. What’s Next • 2010-2011 Full implementation of Standard Monitoring Program • Statewide training webinars on Standard Monitoring Program • Incorporates lessons learned during transition year • Incorporate Data Quality Reports into annual process by review team and provide Technical Assistance to coalitions to implement new reports into their internal quality assurance process • Monitoring trends and risk-based analysis

26. What’s Next • Fraud prevention guidance – including quality interviewing tips on ways to identify and prevent fraud • Early Learning Information System(ELIS) – will continue the monitoring streamlining approach and provide many built-in tools for accountability • Child Care Bureau Internal Control Assessment • Conducted a preliminary assessment of internal procedures • Will begin internal reviews for areas not currently reviewed by the Inspector General’s Office

27. Questions or Requests for Copies of Monitoring Tools Stephanie L. Gehres, CPA Accountability Manager Office of Early Learning, Agency for Workforce Innovation Stephanie.gehres@flaawi.com