Performance Measurement

Performance Measurement For Employment And Training Programs Wagner-Peyser and VETS Reporting

Agenda • Background information • Performance updates • Calculating outcomes on the performance measures • Revisions to existing reporting requirements

General References • ETA TEGL 28-04 (Common Measures) can be found at http://www.doleta.gov/Performance/guidance/tools_commonmeasures.cfm • ET Handbook No. 406 can be found at http://www.doleta.gov/Performance/guidance/wia.cfm

Common Measures Concepts • A participantis someone who has been determined eligible to participate and who receives a program funded-service in a physical location. “Participant”

“Participant” (cont.) • States and grantees are encouraged to consider as participants those individuals who receive services via the Internet and do not access those services through a program-funded physical location.

“Participant” (cont.) • Wagner-Peyser employment services • Universal Access • No formal determination of eligibility is required • DVOP/LVER services • veterans, eligible persons, transitioning service members who meet eligibility criteria

“Self-directed job search” Clarification of exclusion • Application to W-P and VETS: • Exclusion is limited to the act of conducting a job search, job bank browsing or modifying job search requirements. • Some self-directed job search activities should be considered reportable services (e.g. building resumes, accessing workforce information, and enrolling in online job search mini courses).

“Exiter” • Common Measures reporting system calculates performance outcomes based on exit quarter.

“Exiter” (cont.) • Exit occurs when a participant does not receive a service funded by the program or a partner program for 90 consecutive calendar days and is not scheduled for future services. • The date of exit is applied retroactively to the last day on which the participant received a service. • This is a Soft Exit approach.

“Exiter” – W-P and VETS • Significant change from the former labor exchange performance measures: • calculated performance outcomes on a fixed point, based on registration quarter • the first or second quarter following the registration quarter. • Hard Exit.

Exiter Calculation Sample Calculation: Please see handout! It is mathematically possible for an individual job seeker to be a participant and an exiter four (4) times in one year. 4 X 91= 364 Although unlikely, it is possible. Here is a more likely scenario.

Global Outcome Exclusions • Global Outcome Exclusions that apply to WIA Adults, Dislocated Workers, Wagner-Peyser, and VETS • Institutionalization • Health/Medical or Family Care; • Reservist called to active duty; and • Deceased.

Global Outcome Exclusions • New to W-P and VETS • In implementation of global outcome exclusions, states should consider: • High costs of developing system to accept and track supplemental data • Negligible effects on outcome

Reporting Transition from PY 2004 to PY 2005 • States will continue to report receipt of services and outcomes using the rolling four quarters methodology. • For the report due November 14, 2005, States will need to “reach back” to re-calculate number of participants, exiters, and outcome data.

Minimum Data Needed from S-S Participants • At a minimum, data needed from self-service participants: • social security number (SSN) • employment status at the point ofparticipation. • Upon request of a customer’s SSN, the customer must also be offered the opportunity to provide Equal Employment/demographic information.

Entered Employment Rate (EER) Measure: Of those not employed at participation, the percentage of individuals employed in the 1st quarter after program exit. • Includes only those not employed at entry. • Entry to employment is based on wage record data and other administrative data sources. • Employment status at the start of participation is self-reported by jobseekers.

EER – former Labor Exchange measures Common Measures change: • Registrant -to- Participant • Hard -to- Soft Exit • UI Wage Record and other state data system records -to- all of the preceding plus…. supplemental data • States have to weigh benefit of developing follow-up case management system

Employment Retention Rate (ERR) Measure: For those employed in the 1st quarter after exit, the percentage employed in the 2nd and 3rd quarter after exit • Of those employed in the 1st quarter after exit, measure applies to individuals that were not employed at the start of participation, as well as those who were employed at the start of participation.

ERR – former Labor Exchange measures • Change from old labor exchange measure: Employment in the 3rd quarter after exit is new measurement point. • The common measure ERR requires employment in two quarters to count toward employment retention.

Earnings Increase (EI) Measure: Of those employed in 1st quarter after exit, total post-program earnings (in Q2 & Q3 after exit) minus pre-program earnings (in Q2 & Q3 prior to participation) divided by the number of adults who exit during the quarter. • Comparison of pre-program earnings to post-program earnings using dollar amount, not rate. • Includes only those employed in first quarter after exit. • Verified through wage record data only.

EI – Wagner-Peyser and VETS • A new outcome measure for W-P and VETS • Unsure of the expected outcome level • States will report the outcomes results • Outcome goals will not be negotiated for PY 2005 for W-P • PY 2005 will a baseline year for W-P • Negotiation of goals in PY 2006

Workforce Information To integrate employment and workforce information services, WIA Adults, Dislocated Workers, and Wagner-Peyser will collect and report data on the provision of workforce information to job seekers. • Workforce information may be accessed in person through self-help in the resource rooms, individual counseling or career guidance in a One-Stop Career Centers, or remotely via the Internet, and/or in ancillary agency sites.

Workforce Information (cont.) • The W-P employment service program will report on receipt of workforce information as a self-directed and/or staff-assisted service and sub-aggregate outcome data for those who received workforce information. • DVOP and LVER staff members will report receipt of workforce information only as a staff-assisted service.

W-P/VETS and TSMs • Recently Separated Veteran • WIA – time period within last 48 months • W-P/VETS – time period within last 36 months • Transitioning Service Member (TSM) Expects to retire within 24 months or separate within 12 months and seeks employment services. • Homeless Veteran – A veteran who meets the statutory criterion for homelessness. • Intensive Services -Replaces Case Management as a reporting category.

Transitioning Service Members • The Jobs for Veterans Act authorized DVOP and LVER staff members to deliver employment participants is included in the total count and/or services to TSMs. • Information on TSM separately identified on the ETA 9002 A, B, C and D reports and the VETS 200 A, B and, similar to “Recently Separated C reports. • TSM is a reporting characteristic Veteran” and “Campaign Badge Veteran.”

Homeless Veterans • A homeless veteran is an individual who has: • At least one day of active duty service with an “other than dishonorable” discharge; AND • “No fixed, regular, and adequate nighttime residence.” • Services and outcomes for homeless veterans are reported only on the VETS 200 C report.

Intensive Services • The Jobs for Veterans Act replaced the Term “Case Management” with the term “Intensive Services” and referenced Section 134(9)(3) of WIA, which identifies six specific services: • Conduct of assessments; • Development of individual employment plans; • Group counseling; • Individual counseling and career planning; • Case management; and • Short-term prevocational services.

Intensive Services (cont.) • “Intensive Services” has Replaced “Case Management” as a reporting item on the ETA 9002 B and the VETS 200 A-C. • DVOP/LVER staff still must meet the two prerequisites for case management in order to deliver intensive services: • Conduct of an assessment; and • Development of an individual employment plan

Intensive Services (cont.) • After the prerequisites, DVOP/LVER staff can provide: • Group counseling; • Individual counseling and career planning; and • Case management. • Short-term prevocational services are NOT included among the intensive services to be provided by DVOP/LVER staff. • Veterans are not required to receive “core services” prior to receiving intensive services but veterans still are expected to receive standard labor exchange services from Wagner-Peyser staff prior to being referred to DVOP/LVER staff for the delivery of intensive services.

Customer Satisfaction for W-P funded programs Because it is not a common measure, for the Wagner-Peyser-funded employment services, ETA will discontinue capturing: • job seeker customer satisfaction data using the American Customer Satisfaction Index; and • employer customer satisfaction data.

Reporting Job Openings • States should exclude from the ETA 9002E Report those job openings that were initially listed with America’s Job Bank and then imported into the state job bank.

Technical Changes to the ETA 9002 and VETS 200 Reporting For further information, please review the ET Handbook No. 406 or contact your ETA or VETS regional office.

ETA 9002 A Eliminations • Reporting on job seekers who were “Referred to Support Services” (row 17 on previous report). • Column reporting on “Other Race” (column 13 on previous report)

ETA 9002 A Minor Adjustments • Renamed column A from “Total Jobseekers” to “Total.” • Renamed column B from “Employment Status at Registration” and “Unemployed” to “Employment Status at Participation” and “Not Employed,” respectively. • Renamed “Total Active Jobseekers” row to “Total Participants.” • Renamed “Veterans and TSM Job Seekers” row to “Veterans, Eligible Persons, and TSMs.” • Changed age breakout from “Adult (19 and over)” to “Adult (18 and over).” • Changed age breakout from “19-44” to “18-44.”

ETA 9002 A Additions • Reporting row: Total number of participants who “Received Workforce Information Services.” • Reporting row: “Total Exiters” during the reporting period.

ETA 9002 B Eliminations • All column level reporting by age breakouts for each category of veteran. On the previous report, this included column B, rows 5-7, column C, rows 9-11, column D, rows 13-15, and column E, rows 17-19. • Reporting requirements for veterans who were “Referred to Support Services” (row 9 on previous report).

ETA 9002 B Minor Changes • Renamed column A from “Total Veterans and TSM Jobseekers” to “Total Veterans, Eligible Persons, and TSMs.” • Changed age breakout in column A from “19-44” to “18-44.” • Switched columns B and C. On new report format, column B is labeled “TSMs” and column C is labeled “Campaign Badge Veterans.” • Renamed column F from “Newly Separated Veterans” to “Recently Separated Veterans (3 yrs).” • Renamed row 1 from “Total Active Veterans and TSM Jobseekers” to “Total Veterans, Eligible Persons, and TSMs.” • Renamed, where appropriate, “case management services” to “intensive services.” • Renamed, where appropriate, the term “placed in” with “entered into.”

ETA 9002 B Additions • 3 reporting rows that include the total number of veterans served who were ages “18-44,” “45-54,” and “55 and over” for each category of veteran. • Reporting row that includes the total number of veterans who “Received Workforce Information Services.” • Reporting row that includes “Total Exiters” during the reporting period.

ETA 9002 C Eliminations • All column level reporting based on “Employment Status at Registration” (column B on previous report). • Column reporting on “Other Race” (column 13 on previous report) • Reporting requirements for job seekers who “Entered Employment Following Staff-Assisted Services” (row 11 on previous report). • Row reporting requirements for the job seeker and employer customer satisfaction measure (rows 12 – 17 on previous report).

ETA 9002 C Minor Adjustments • Renamed column A from “Total Jobseekers” to “Total Exiters.” • Included, where appropriate, the term “numerator” and a denominator” for the entered employment and employment retention measures. • Changed age breakout from “19-44” to “18-44.”

ETA 9002 C Additions • Reporting row to capture “Six Months Earnings Increase.” • 3 reporting rows to capture common measures outcomes for participants who receive workforce information services. • “Entered Employment Rate Follow Work-Info Services,” • “Employment Retention Rate Follow Work-Info Services,” and • “Six Months Earnings Increase Follow Work-Info Services.”

ETA 9002 D Eliminations • All column level reporting by age breakouts for each category of veteran. On the previous report, this includes column B, rows 5-7, column C, rows 9-11, column D, rows 13-15, and column E, rows 17-19.

ETA 9002 D Minor Changes • Changed age breakout in column A from “19-44” to “18-44.” • Switched columns B and C. On new report format, column B is labeled “TSMs” and column C is labeled “Campaign Badge Veterans.” • Renamed column F from “Newly Separated Veterans” to “Recently Separated Veterans (3 yrs).” • Included, where appropriate, the term “numerator” and “denominator” in the rows for the entered employment and employment retention measures.

ETA 9002 D Additions • 3 reporting rows that includes the total number of veterans who entered employment for each category of veteran according to the following age breakouts: “18-44,” “45-54,” and “55 and over.” • Reporting row to capture “Six Months Earnings Increase.” • 3 reporting rows to capture common measures outcomes for jobseekers who receive workforce information services. • Entered Employment Rate Follow WI Services, • Employment Retention Rate Follow WI Services, • Six Months Earnings Increase Follow WI Services

ETA 9002 E • No changes - Everything stays the same! 

VETS 200 A, B and C Reports Eliminations Services Provided • Reporting on veterans who were “Referred to WIA Services” (row 11 on previous reports). • Reporting on veterans who were “Referred to Support Services” (row 12 on previous reports). Results and Outcomes • The 3 rows of reporting for the entered employment data: “Entered Employment,” “Entered Employment Rate Base,” and “Entered Employment Rate” (rows 17-19 on previous reports). • Placeholder for state reporting on Job Seeker Satisfaction Score (row 32 on previous reports).

VETS 200 A, B and C Reports Additions • Reporting row that includes the “Total Exiters” during the reporting period. Services Provided • Reporting row that includes the total number of participants who “Received Workforce Information Services.” Results and Outcomes • Reporting row that includes an aggregate count of the “Six Months Earnings Increase” during the reporting period.

VETS 200 C - Only Additions • Reporting column on the VETS 200 C only that includes aggregate reporting on services to “Homeless Veterans” (column I).

SAMPLE CALCULATION of EXITER • An unemployed job seeker came to the One-Stop Center on July 1, 2005 and browsed the job bank. She returned on July 3rd and created and posted a resume. On July 5th she modified job bank search requirements and accessed workforce information. She did not receive any more services for 90 days. Employed but interested in looking at other options, she returned to the One-Stop on January 5, 2006 to update her resume and look at workforce information. She did not receive any more services for 90 days. She remained employed through January 2007. Questions: • What is her participation date(s)? • What is her exit date(s)? • What quarters do you follow for entry to employment? • What quarters do you follow for retention? Earnings gain?

July Aug. Sept. Oct. Nov. Dec. Jan Feb. March April May June July Aug. Sept. Oct. Nov. Dec. Jan2005 2006 2007 July 1: Browsed Job Bank July 3: Created Resume – Participation Date 1 July 5: Modified search requirements. Looked at workforce information. Oct. 3: 90 days have elapsed, no service. Exit program. Exit date 1: July 5, 2005. (Exit qtr July – Sept. 2005) Oct. – Dec. 2005 qtr is the first qtr after Exit date 1. Look at this qtr for entry to employment for participation 1. In this case the participant was found to be employed Jan 5: Updated resume and accessed workforce information – Participation Date 2. (still employed) April 5, 2006: 90 days have elapsed, no service. Exit program. Exit date 2: Jan 5, 2006 (Exit qtr 2 Jan – March 2006) April-June 2006 qtr is the first qtr after Exit Date 2. Because jobseeker was employed at this participation, not included in the EER for participation2. Jan- March 2006 and April-June 2006 are qtrs 2 and 3, respectively, after exit of participation 1. Because participant was employed in first qtr after exit 1, look at these qtrs for Retention and Earnings Increase. July-Sept. 2006 and Oct.-Dec. 2006 are qtrs 2 and 3, respectively, after exit of participation 2. Because participant was employed in first qtr after exit 2, look at these qtrs for Retention and Earnings Increase.

Performance Measurement