An Evaluation of the Individual Training Account & Eligible Training Provider Demonstration

1. An Evaluation of the Individual Training Account & Eligible Training Provider Demonstration Preparedby: Social Policy Research Associates Under Subcontract to: Mathematica Policy Research, Inc.

2. BackgroundThe Demonstration Project • About the ITA/ETP Grantees • 13 grantees (states and LWIAs) funded by USDOL for ITA/ETP system development • About the Evaluation • Two rounds of multi-day site visits to 28 local workforce areas in 13 states • Discussions with grant managers, line staff, administrators, vendors & ITA-holders • Final Report issued Dec 2004(http://www.doleta.gov/reports/searcheta/occ

3. Policies Regarding ITAs:State/Local Emphasis on Training • Limited use of training at the start of WIA • Start-up issues and confusion about ITAs • Low demand for training in hot economy • Training levels have since rebounded, and ITAs are strongly embraced • Some local areas continue to see a strong role for contract and customized training

4. Policies Regarding ITAs:Helping Customers Make Training Choices • Emphasis on customer choice predominates • Flexibility in how customers move through service levels • Case managers act as facilitators • Sites promote an “informed choice” model • Some assessment and/or field research on training occupations and vendors is nearly always required of ITA holders

5. Policies Regarding ITAs:Limits on Customers’ Choices • Median cap on ITA amount is about $5,000 • Yet caps vary widely across local areas • Low of about $1,700 • High of $10,000 • Time limits are usually 1 or 2 years • Different ways of defining “demand occupations” • Reliance on formal lists • Local counselor discretion

6. Developing the ETP ListRequirements & Procedures for Eligibility • Aggressive efforts to recruit vendors • Procedures to streamline application process • Having vendors submit to a state clearinghouse • Allowing for electronic submissions • Few requirements for initial eligibility, beyond state licensure • Substantial start-up and on-going costs

7. Developing the ETP ListRequirements for Subsequent Eligibility • The start of subsequent eligibility was often deferred • Performance requirements for vendors for subsequent eligibility vary widely, in: • Number of measures to meet • Thresholds on the individual measures

8. Developing the ETP ListDefinitions and Data Capture • Definitions of key terms vary markedly or are vaguely defined • e.g., what counts as a program, a completer, employment • Procedures for data capture also vary • Most states rely on centralized UI matching for measuring employment outcomes • Some also attempt matches with post-secondary enrollment files to measure program completion • But some states require vendors to self-report all items

9. Quality of the Consumer Report System • Dramatic progress in developing CRS • CRS systems vary in sophistication & ease of use • Beyond basic searches on program type & vendor location, some allow searches on cost, whether ITA approved, accessibility, etc. • Beyond basic program/vendor info, some provide much more (e.g., class sizes, waiting periods, admission requirements, teaching method, etc.) • Performance data are usually very spotty • At least at the state level, there is usually ample choice • Public institutions generally constitute a minority of vendors but a majority of programs • Choice is often very limited in rural areas

10. Vendor Reactions • Public institutions • Provide high-quality, low-cost training resource • Many balk at the ETP performance and reporting requirements, as ITAs constitute a tiny part of their business • Many perceive their mission broadly and eschew a narrow focus on employment outcomes • Proprietary schools • Generally embrace the ITA system; very market driven and some aggressively pursue ITAs as a source of new business • Are generally very focused on employment outcomes • Community-based organizations • Have often relied heavily on contract training • Are finding it hard to compete in an ITA system • Low capitalization puts them at high risk

11. Summary and Conclusions • ITA Issues • Use of training has rebounded from initially low levels • ITAs are strongly embraced • Substantial local-area variability in ITA policies • A focus on customer choice and empowerment is very evident • An “informed choice” model predominates • Some local areas continue to see a strong role for contract and customized training

12. Summary and Conclusions (concluded) • ETP Issues • Some vendors balk at ETP requirements, undermining customer choice • Eligibility and data requirements for vendors vary enormously • Consumer Report Systems • Dramatic progress in on-line systems • Performance data are spotty