Program and Compliance Management

1. Program and Compliance Management V I R T U A L L Y Workshop: Documentation Desires, Duties, Dilemmas

2. Outline • Introduction • Eligibility vs. Data Validation • DV Documentation • Documenting Common Measure Outcomes • Best Case Scenario?

3. Introduction

4. First Things First Documentation is not just paper(Utah, for instance, is entirely paperless) Requirements differ across programs and populations and pertain to: Who you’re serving …documenting characteristics What they’re getting …documenting services With what results …documenting outcomes “THEY said…” – Make sure what you think is required really is

5. Potential Data Sources

6. The Goldilocks Factor Not too much. Not too little. Just right. Supports stewardship, accountability Leaves audit trail Consistent with data validation requirements

7. Issues from Data Validation Reviews Lack of documentation or insufficient documentation to support service provision (particularly for youth services) State and/or Local policy includes incorrect sources Issues with case files and case notes >>

8. Issues from Data Validation Reviews (2) Case Files Can’t be located, documents missing Inconsistencies between file and MIS Lack of consistent file structure One file for multiple periods of participation Case Notes Quality (insufficient, illegible, irrelevant); Missing (where needed or required by policy) Case notes contain info that differs from other documents or the MIS (e.g., different dates)

9. Eligibility vs.Data Validation

10. Are the Following Statements TRUE or FALSE? Federal policy elaborates on the eligibility requirements contained in the statute and regulations. Documentation for eligibility and documentation for data validation are the same.

11. EXAMPLE: Youth Eligibility • Sec. 101(13) of the WIA states that eligible youth are low income and have at least one barrier: • Basic Skills Deficient • School Dropout • Homeless, Runaway or Foster Child • Pregnant or Parenting • Offender • Youth who requires additional assistance… Documentation Disconnect

12. BSD is defined at Sec. 101(4) as being determined by a “generally accepted standardized test…” …So eligibility is based on a standardized test but Federal DV policy has school recordsand case notesin addition to standardized tests (TEGL 28-11, May 9, 2012 (page A-18)) Basic Skills DeficiencyDocumentation Disconnect

13. AND… if the youth is BSD as defined, then Literacy/Numeracy requirements must also be followed …which means the ‘standardized test’ has to be acceptable for L/N or another test administered …which means reporting several WIASRD data elements and following the associated DV documentation requirements Basic Skills DeficiencyDocumentation Disconnect (2)

14. The 6th Youth Eligibility CriterionDocumentation Disconnect “…requires additional assistance to complete an educational program or to secure and hold employment as defined by State or local policy. If the State Board defines a policy, the policy must be included in the State Plan.” Many states don’t define, leaving it up to locals How is it defined in your state?

15. DV reviews have cited more than lack of documentation for this eligibility item Including insufficient case notes to validate Area of Concern often included in final report recommending a thorough review of local definitions/policy on this item (e.g., too vague) The 6th Youth Eligibility CriterionDocumentation Disconnect (2)

16. Data Validation Documentation

17. Types of Allowable Source Documentation The document itself (e.g., certificate) State MIS (management information system) Cross-Match Self-Attestation Case Notes

18. “State MIS” Specific, detailed information stored in the state’s management information system that supports an element An indicator, such as a checkmark on a computer screen, is not acceptable source documentation in and of itself A deeper level of inquiry is required

19. Example of Using “State MIS”Validating Date of Exit After the exit date is noted in the state MIS, what additional information would you look for in order to pass this data element? The service provided and last date received No other qualifying services provided within 90 days of the last service Individual not scheduled for future service and has no gap in service

20. “Cross-Match” Requires accessing a non-WIA database to find detailed supporting evidence for the data element An indicator or presence of an SSN or checkmark in a non-WIA database is not sufficient evidence TANF participation can be determined by a cross-match with the state’s public assistance database but it is not sufficient to just find the SSN in the database

21. Example of Using “Cross-Match”Validating Youth Placement A state uses a cross-match (computerized record) with their Higher Education Administration as the source document for verifying placement into post-secondary education What information would you need to pass this data element? Name or some other personal identifier Information showing enrollment in post-secondary education Date that indicates the youth was enrolled in first quarter after exit quarter

22. “Self-Attestation” When a participant states his/her status for a particular data element, then signs and dates a form acknowledging this status Key elements for self-attestation: Participant identifying his/her status for permitted data elements Signing and dating a form attesting to this self-identification

23. Example of Using “Self-Attestation”Validating Displaced Homemaker Status What type of information/language must be included on a self-attestation form to be considered an acceptable source document for this data element? The individual has been providing unpaid services to family members in the home, and Has been dependent on the income of another family member, and Is no longer supported by that income, and Is unemployed or underemployed, and Is experiencing difficulty obtaining/upgrading employment. Signature and date

24. “Case Notes” Paper or electronic statements by the case manager that identify, at a minimum, the following Participant’s status for a particular data element, Date the information was obtained (and from who, as appropriate) Name of case manager

25. Example of Using “Case Notes”Validating Other Reasons For Exit What type of information must case notes include? Participant information (likely moot point given electronic case notes) Individual will no longer be able to participate due to [state reason*] *Institutionalization; Health/Medical; Deceased; Family Care; Reservist called to Active Duty; Relocation to mandated facility for youth

26. Documenting Common Measure Outcomes

27. Data Sources According to TEGL 17-05, these are the five basic data sources used to demonstrate common measure outcomes Wage Records WRIS, FEDES Administrative Records Assessment Instruments Supplemental Data

28. Documenting OutcomesUsing Supplemental Data In the absence of wage records, can be used to demonstrate employment and retention only (i.e., not earnings) All data and methods are documented and subject to audit “Allowable sources. . . include case management notes, automated data base systems, One-Stop operating system’s administrative records, surveys of participants, and contact with employers.” (TEGL 17-05, page 26) The data and methods to obtain it can be clarified in state policy guidance (e.g., maintaining correct contact information is job one)

29. Documenting OutcomesUsing Supplemental Data (2) Some states use supplemental data more than others (and some states don’t use it at all) SD and UT do not use supplemental data For the first quarter of PY12, supplemental data accounted for what percentage of the numerator of the Adult EER? MT – 6% ND – 2% WY – 9%

30. Documenting OutcomesCommon Measures Entered Employment Rate Employment Retention Rate Average Earnings Placement in Employment or Education Attainment of Degree/Certificate Literacy/Numeracy Wage Records WRIS, FEDES Supplemental Data Administrative Records Assessment Instruments According to TEGL 17-05, these are the five basic data sources used to demonstrate common measure outcomes (A—E). Question: Which common measure utilizes four of the allowable data sources?

31. Best Case Scenario

32. The Ideal ? Use federal DV source documentation as foundation State provides additional guidance to clarify federal sources where necessary State provides guidance on “other” documentation to ensure statewide consistency Local Boards/One-Stop Operators provide additional guidance where needed (consistent with State and federal)

33. The Ideal ? (2) • Irrespective of level, documentation policies and procedural requirements are part of staff orientation and general data management • Consistency • Accountability