Presentation Sept 8, 2016 Richmond, VA

1. A Randomized Controlled Trial to Improve Wage Outcomes of SSDI Customers of VR: Experiences of Minnesota and Kentucky VR Presentation Sept 8, 2016 Richmond, VA

2. Presenters • Susan Foley, Institute for Community Inclusion • Allan Lunz, Minnesota DEED-VRS • Kellie Scott, Kentucky OVR • Todd Honeycutt, Mathematica Policy Research • Joseph Marrone, ICI

3. What is the SGA Project? • Model Demonstration • SSDI only non-blind VR customers • Improve earnings outcomes at closure • Innovation must be within the control of the VR agency.

4. Funding from the Rehabilitation Services Administration Rob Groenendaal Jerry Elliott Project Officers

5. Partners • Mathematica Policy Research • Michael D’Alto • VR Development Group • Vendors in KY and MN providing financial specialist services

6. The Innovation! Early coordinated team approach (CTA) to rapidly engaging SSDI only applicants in the VR process to improve earnings at closure of VR services. • CTA includes job placement, VR counselor and financial specialist with customer Pre-IPE and then throughout VR tenure. • Number of days specified for each rehab process step.

7. How did we get there? • Intensive site visits with 8 state VR agencies • RSA 911 Data Analysis • Expert opinion through Delphi Panel • Document review • Literature review

8. Few key findings of early process of creating innovation • Agencies that had above average wage outcomes for SSDI only customers had: • Fewer counseling staff and more counseling support staff as a % of all staff • Had fewer customers on wait lists • Provided more services to customers through VR personnel and purchased fewer services in aggregate from vendors • Had fewer average days to IPE • Had fewer SSDI only customers closed because of refused services/uncooperative • Had lower average cost of purchased services for SSDI only customers who received services.

9. Case Study Findings • Multiple SVRAs emphasized early and progressively detailed access to financial services • Rehabilitation counseling strategies with a sense of urgency and rapid engagement. Momentum is good customer service. • A business relations focus to job development rather than supply side job placement. • Active attempts to align office level procedures and policies for responsive client services.

10. Recruiting the SVRAs • Identified SVRAs with sufficient numbers to host project (n = 19) • Had to eliminate those SVRAs that: • Too few number of offices • Were closed to all categories (OOS) or had lengthy weight lists • Had intense political, budgetary, union or reorganization issues • Purchased most of its services or moved to use of “virtual” employees • Some states elected to go for PROMISE grants • Made site visits to six SVRAs

11. SGA Project-Minnesota Allan LunzDEED: Vocational Rehabilitation Services

12. Why Minnesota said “Yes” to the SGA Project • Rapid Pacing • Teaming Approach • Engagement • Additional Program Income

13. Preparations for the Future • Workforce Innovation Opportunities Act(WIOA) • Minnesota’s Olmstead Plan • Ongoing Relevance of the State VR Program

14. Keys to Successful Implementation • Top level leadership support  • Buy in at local level from RAMs and VRCs • Added resources available to SSDI clients on treatment teams . 

15. With Challenge comes Success! Immediate Feedback Value of: • Team Approach • Financial Education • Up Front Resources & Support • Pacing

16. Kentucky’s SGA Project Kellie Scott, Program Evaluator Kentucky Office of Vocational Rehabilitation

17. Why Was KY OVR Interested in the SGA Project? • Building capacity in work incentive counseling; • Fit in with recent asset development initiatives; • Leadership interested in enhanced pacing; • Possibility of increasing positive employment outcomes and SSA reimbursements; • Previous positive experience with ICI; • Progressive Leadership (David Beach was Director).

18. SGA PROJECT KWICs Ft. Wright*-#16 Carol Leonhart Florence-#8 Ryan Henson Bluegrass-#14 Brad Mills Donna Mundy – Addis Megun – Evangeline Johnson - Middletown- #15 Jenny Lampton Ashland*-#12 Brent Sturgill Lexington*-#9 Jonathan White Kenton Boone Campbell Louisville* -#5 Greg Willett Gallatin Grant Pendleton Bracken Carroll Elizabethtown-#6 Jim Pike Trimble Mason Owen Lewis Greenup Robertson West Liberty-#10 Tony Perry Harrison Henry Owensboro*-#3 Jason Cole Oldham Fleming Nicholas Boyd Scott Carter Franklin Shelby Bourbon Jefferson Rowan Bath Elliott Woodford Lawrence Fayette Spencer Montgomery Anderson Clark CDPVTC Barb Pugh Bullitt Menifee Morgan Jessamine Meade Hancock Mercer Powell Johnson Nelson Martin Paducah-#1 Lisa Allen Henderson Breckinridge Madison Wolfe Magoffin Washington Estill Daviess Hardin Union Garrard Boyle Floyd Lee Breathitt Marion McLean Larue Webster Pike Ohio Lincoln Grayson Owsley Jackson Rockcastle Knott Crittenden Hopkins Taylor Perry Hart Casey Livingston Green Butler Edmonson Clay Muhlenberg Leslie Letcher Laurel Pulaski Caldwell Adair Whitesburg-#13 Viva Anderson Ballard Lyon McCracken Russell Warren Barren Metcalfe Christian Knox Logan Carlisle Wayne Harlan Marshall Cumberland Whitley Trigg Todd Monroe Bell Clinton McCreary Graves Simpson Allen Hickman Calloway Fulton Bowling Green*-#4 Missy Wheeler Madisonville-#2 Vacant Danville*-#7 Ruth Dyer

19. What were the barriers? • Lack of local staff buy-in; • Bureaucratic inertia; • Changes in staff and administration; • Lack of work incentive capacity; • Distractions for leadership (WIOA implementation, budget issues, etc); • One more thing for VRC’s to “get their head around (e.g. large caseloads); • Lack of infrastructure (should have spent 1st year updating PC’s, online assessments, DB101). 20/20 hindsight

20. Successes • Leadership and staff see value in participation of KWIC in team, early involvement of job placement and enhanced pacing; • Quick and early identification of SSA recipients; • Able to build capacity in several areas; specifically Job Placement; • Preliminary indications of increased employment rate among SSDI recipients

21. SGA Project Evaluation Todd Honeycutt Mathematica Policy Research Center for Studying Disability Policy thoneycutt@mathematica-mpr.com

22. SGA Project Evaluation Design • Examine KY & MN projects separately, based on state’s own implementation model • Implementation evaluation • To what extent did offices implement their assigned conditions with fidelity to their design? • What are the key lessons learned from implementation experiences? • Outcome evaluation • What impact did the innovations have on VR service delivery and client employment outcomes?

23. Preliminary Evaluation • Assess processes and outcomes for SSDI-only clients who applied through January 2016 (KY – 521 clients; MN – 645 clients) • Data include • RSA-911 data • Site visits • Interviews conducted with staff involved in the demonstration • Agency administrative data • Preliminary results shown

24. Kentucky Qualitative Themes • SGA Project model represented a substantial shift in priorities and operating procedures • Challenging for counselors to change practice patterns related to presumptive eligibility • Counselors enjoyed increasing the pace and expressed a positive attitude toward the team approach • Difficulty recruiting enough financial counselors • Difficulty with follow-up team meetings

25. Kentucky Enhanced Services Relative to Usual Services • Higher eligibility rates (98% vs. 94%) • Shorter median time to eligibility (3 days vs. 30 days) • Higher proportions obtaining IPEs (71% vs. 60%) • Shorter median time to obtaining IPEs (42 days vs. 77 days) • At six months after application, no difference in the proportion receiving services (50% vs. 55%) • But more likely to have exited with employment (17% vs. 8%)

26. Minnesota Qualitative Themes • Adopting the new models was initially a challenge • Increased pace of service could be adopted more broadly with the right supports • Greater access to financial and benefit information enhanced the VR experience • Perceived as less effective and harder to implement in sites serving large geographic and mostly rural areas

27. Minnesota Implementation Fidelity • Enhanced services group typically did not achieve program milestones • 78 percent received IPE, but only 37 percent received IPE within 30 days • 14 percent received a benefits analysis, with almost all occurring within 8 weeks • Over half received a team meeting, and only 23 percent of meetings happened within 7 days • Implementation varied across offices

28. Minnesota Enhanced Services Relative to Usual Services • Shorter median time to eligibility (3 days vs. 7 days) • Higher proportions obtained IPEs (78% vs. 68%) • Shorter median time to obtaining IPEs (32 days vs. 46 days) • At six months after application, less likely to be still receiving services (55% vs. 64%) and more likely to have exited with employment (16% vs. 10%)

29. Evaluation Next Steps • Release preliminary evaluation reports (Fall 2016) • Conduct full evaluation for all enrollees, including early earnings impacts (Fall 2017)

30. LESSONS LEARNED • LEADERSHIP SUPPORT AS SGA AS AN “IMPORTANT” PIECE FOR VR AGENCY QI, NOT JUST AN EXTRANEOUS PROJECT • PACING DID NOT SCARE AWAY CLIENTS NOR DISRUPT THE AGENCY • EARLY INVOLVEMENT OF FINANCIAL SPECIALISTS AND EMPLOYMENT STAFF VERY HELPFUL – ESPECIALLY THE FINANCIAL DISCUSSION (NOT JUST BENEFITS COUNSELING) • LISTEN TO CLIENTS ON HOW THE TEAM SHOULD BE ROLLED OUT- THEIR TIME IMPORTANT TOO

31. LESSONS LEARNED • STAYING FLEXIBLE WHILE ADHERING TO CORE INTERVENTION DESIGN • ICI TECHNICAL ASSISTANCE USEFUL BUT LOCAL TA (E.G., MN DSHS) JUST AS IMPORTANT • ISSUES THAT DID OCCUR OFTEN TRACED BACK TO BROADER MGMT QUESTIONS MORE THAN THE SGA PROJECT DESIGN OR IMPLEMENTATION ITSELF • MAKE IT FUN, NOT JUST A CHORE