Why a Performance management System for WIOA : And what makes it customer service

1. Why a Performance management System for WIOA:And what makes it customer service Clinton Flowers, Tina Bonnot, Performance Research Workforce Development Specialist Missouri Division of workforce development

2. Overview • What are the performance measures for WIOA? • How is the State currently performing using these measures? • What are the components of a performance management system? • What are the customer service features surrounding each measure? • What is known about data capture that can explain case manager behavior? • How is the data from reported performance outcomes used? • What are circumstances for performance failure? • Feedback for Learning………..Your Questions?

3. Performance measures • Accountability measures Seven accountability measures: 1) Employment after (Q2) 6 months; 2) Employment after (Q4)12 months; 3) Median earnings (Missouri still captures average earnings in addition to median); 4) Credential attainment; 5) Measureable skill gain; 6) Effectiveness in Repeat Business services; 7) Effectiveness in Market Penetration. National research has suggested “common measures” across all partner programs could be Entered Employment, Employment Retention, and Median Earnings.

4. What are the customer service features of the employment measure: Did they get a job? • First, properly record the information. But the essential nature is, getting them a job? • What shall we do to ensure that? • Screening, are the job ready or in need of remediation? Assessment? • Do they know the job they want? Have and can use Labor Market Information? • If “keeping” the job is good, doe we have knowledge the placement is not a high turnover?

5. What are the customer service features in the earnings measure: How much? • Does the customer have realistic expectations for their earnings? • Have they considered skills used for higher earnings in other jobs? • Do they need more skills first? • Are they provided Labor Market Information to help get higher earnings.

6. What are the customer service features of credential Attainment? • While considering referral to training, has there been a review of expectations for the credential? Time, effort, earnings expectations? • Is there a business model for an agreement with the training provider to ensure the credential is reported? • Is there agreement on Measureable Skills Gains documentation, before the credential is attained?

14. Five components • Data capture- GSI/MoJobs, MoPerforms, WIPS • Performance Measures (national study – common metrics) • Reporting - State and Local Plans - Annual Report - Accountability, Influence, Incentives • Federal Reporting • Evaluation

15. Data capture • Geographic Solutions Inc. is the firm vendor of the database/case management system. The firm is alternatively known as GEOSOL, at DWD performance we call the firm GSI. • The firm GSI allows state clients to name their individualized version of the software, from which we call it MoJobs. Obviously, GSI is the background, MoJobs is the front door • MoPerforms is also a client named software, the vendor is FutureWork Systems. • The Workforce Integrated Performance System (this is WIPS). Federal edit check and submission software.

16. DATA FLOW The Flow of Data Origin to Destination The database is the source, complete record. Contains many static reports, but primarily about affect, services, and processing of clients + Federal reports. For assessing and making business decisions about whom and how they are served, you need analytics + State oversight. Also edit and correct for WIPS. Workforce Integrated Performance System – WIPS is edit check software that helps correct/minimize errors. While there are now only seven federal performance measures for accountability, the analytic software enables us to create many more, like organizational health, process, impact and outcomes useful for program evaluation.

17. Performance measures • In reporting, our workforce program participants are enrolled/registered. If not completely, or properly enrolled, the less robust or accurate the outcome data. • We measure certain steps along the way to great outcomes, but outcomes are calculated on exiters. This means they are not still in the program they participated in, they did not leave the program for health/or death, justice intervention or military duty, they exited. • All outcome measures are exit based, how did they succeed, or not, after services. Based on people the system is no longer serving. Exiters!

18. Performance Measures • If some measures are exit based, then as much as we have been accustomed to not performing follow-up, to some degree, we must go back to that system. • For example, enrolling someone in training is a success, but that is not an exit, right? In order to be a success in performance, they must receive a credential recorded in the correct field in the case management system (MoJobs) to count as a success. • Did you realize, that for Youth Program participants, that attain a high school diploma/equivalency, they must also be employed or be enrolled in an education/training within one year after exit for that credential to count as a success? • Next Generation also created large numbers of persons “self-attested” to be in an educational program, for which we cannot now confirm a credential is achieved.

19. About serving employers • Effectiveness in serving employers is about contacting and managing business services. It is similar to case management of employers for your business services team. • Repeated services to business: Calculated by looking back three years, at all of the businesses that have been provided workforce development services, how many received more than one—a repeated service. Expressed as a numerator-denominator and rate. • Market Penetration rate: Calculated by establishing the known number of establishments in Missouri (or a subset within each region) and counting how many of them have been provided a business service. Expressed as a numerator-denominator and rate.

20. Vitally important for Effectiveness Measures • The list of services that count for these measures is extensive so it is generous, almost any business contact counts, ranging from providing labor market information, to explaining program services. See DWD Issuance 22-2017 – April 4, 2018. • Here the is the most important take-away about these employer service measures: Staff must record these services in MoJobs case management system, that means aligning the work of your employer/business services team and case managers to ensure the data is entered so the work and services you have provided get calculated and performance is achieved!

21. If A PERFORMANCE Measure is not met? • State implications in WIOA Section 116(f)(1)(A): If a State fails to meet…..(a measure’s negotiated goal) for any program year, the Secretary of Labor and the Secretary of education shall provide technical assistance, including the development of a performance improvement plan. • In subsection (B) of Section116: If such failure continues for a second consecutive year, (the Governor’s set aside 15% allocation is reduced by 5%). This varies by allocation changes per year, but may be as much as just under $500,000.

22. If A PERFORMANCE Measure is not met? • Local Implications in WIOA Section 116(g)(1): If a local area fails (a negotiated measure goal) for any program year ---technical assistance is provided. • However, in subsection (g)(2) of 116: If a local area fails (a negotiated measure goal) for a third consecutive year, the Governor shall take corrective action, such as: • - 1 Require the appointment of a new Board; • - 2 Prohibit the use of failed training providers; or, may • - 3 Reorganize the local board and its geography. A nuclear option! • A completed sanction process is not yet fully developed by USDOL/Fed. Education

23. Performance Observations • The number one reason a performance measure is failed, there are not enough people placed in the measure? • The system is not historically accustomed to effectively tracking credential attainment. • Credential attainment and verification is more difficult than congress intended, however, training provider contracts should prevent failure. • Current data does indicate some lack of follow-through. • Current data may mean lack of knowledge to result the data, kept in case notes. • Youth credential is typically concerned with a fickle participant. • However, data show higher success with youth in programs, less so training.

24. Credential Attainment Strategies • A special initiative to increase employee awareness. Continuous improvement training. • A priority to train JobStat staff. • Train case management staff. • Train all other stafffor awareness.

25. Business decision metrics • GSI – counting things, events, characteristics, and process metrics, often for case managers. For example: How many persons received services last quarter? • MoPerforms – Cross correlation of data. For example, how single women were served last quarter, and how many of them were African American females, and aged 20-45? • WIPS – i.e., If Date of Program Entry (WIOA) (PIRL 900) is NOT blank, THEN School Status at Program Entry (WIOA) (PIRL 409) must NOT be blank, edit and correct 5,103 records showing a blank in (PIRL 409) School Status at Program Entry. For example.

26. Organizational Health metrics Under consideration • Time to Exit – other time based indicators, cycle time. • Work-based Training – because these are the highest impact trainings. • Average Earnings – can any policy be associated with time periods as cause/effect. • Return on Investment. What is the ratio of earnings generated to dollars expended? • Job Orders completed vs. Job Orders closed (premised upon working job orders). • Traffic (assessed as staff assisted). The traffic that matters the most is staff assisted.

27. Evaluation • WIOA Sec. 116(e) requires the evaluation of state programs. “The State, and the State agencies responsible….for the core programs….shall conduct ongoing evaluations of activities carried out under such programs.” Requires both state and local board involvement. • Comparative impact questions. “Did persons who received a CRC have better outcomes than those that did not?” Turns out they do………. proven in an Arkansas study. • Comprehensive comparative evaluation: control group, experimental group, expressed impact to be assessed. Did the persons given the service or actions intended to impact receive the impact, and how does that compare to the control group that is similar but did not receive impact services or actions. • Finally, there is propensity matching. Common in service business like workforce.

28. rEPORTING • Federal performance reports. Local areas collaborate to meet the State expected levels of performance or meet the consequences of failure. • Federal reports are due submittal quarterly, 45 days after the end of the calendar quarter. • The WIPS portal is opened for submittal of Program Year data by September 1st each yr. • The data may be cleaned, edited, corrected, until the final end-of-year data submitted with narrative as the Annual Report, due by October 1st each year. • Other data are used to make business and policy decisions about how customers are processed, types of services having an impact, conducting evaluation or measuring organizational health. • Friendly competition is used to incentivize behavior. A little sunshine functions like a little challenge.

29. State and Local plans • State Plans are required by law to contain in writing the explanation and description of an assessment of the social and economic conditions of the area, focused on explaining the driving economic sectors, and how the strategies and programs used by the agencies involved are designed to meet the deficiencies or needs of the State. • As for a state performance management system, state plans are meant to assess, explain, discuss, in writing how the activities meet the needs. They are posted with expected levels, and then negotiated levels of performance. • Accountability is a primary purpose of State plans, as well as credibility.

30. local plans • Local Plans mirror the State plans. • Both State and Local plans provide credibility, accountability, influence, incentives shape the workforce system. • Innovation is needed and is being implemented. Including Regional Planning.

31. Essential TAKE-AWAYS on PERFORMANCE for job center management • The number one reason any performance measure is failed: There are not enough people in it! Get people jobs, help them keep jobs, with good earnings, and guide them to a measureable skill gain and/or a credential. • Establish a relationship with customers. It is essential that we have a meaningful exchange of understanding, “This is what we can do for you. Please do something for us, call a case manager when you get a job, that degree or any success based on your visit with us today!” Like any business, we can improve when we measure our success! • Connect with the actions in MoJobs for these measures. If it is not in data entry, it is not in performance credit!

32. Essential TAKE-AWAYS on PERFORMANCE for job center management • Consider some method for follow-up for Youth Credential attainment and to also determine if they are employed. • Consider same for earnings. If you learn feedback from DWD, but you know someone in fact did get a job, exercise supplemental wage data system in MoJobs.

33. We have overviewed Five components • Data capture- GSI/MoJobs, MoPerforms, WIPS • Performance Measures (national study – common metrics) • Reporting - State and Local Plans - Annual Report - Accountability, Influence, Incentives • Federal Reporting • Evaluation YOUR QUESTIONS?