National Dissemination of an Evidence-Based Practice: Supported Employment in VHA

1. National Dissemination of an Evidence-Based Practice: Supported Employment in VHA Sandra G. Resnick, Ph.D. VA Northeast Program Evaluation Center, New England MIRECC, and Yale University School of Medicine

2. Supported Employment • Principles • Goal is competitive employment • Zero-exclusion • Rapid, individualized job search • Long-term follow along support • Integration with Mental Health Team • Considered an EBP • Consistent findings in RCTs • Well defined model • Validated fidelity scale

3. VACO – NEPEC Partnership • Legislation passed/money allocated in 2003 • Funds provided to VACO • Clinical initiative • NOT research • NEPEC • CWT monitoring since 1995 • Experience in dissemination of innovative practices • Decisions made in partnership: • VACO: all sites will have SE (~130 sites) • NEPEC: centrality of fidelity monitoring • VACO supportive of QUERI grant application

4. Implementation Plan: Phase 1 • “Mentor-trainer” sites • One per VISN (21) • Money for a total of 3 – 4 FTEE • 1 Mentor Trainer (MT) • Non-MT sites • 1 FTEE • Trained by MT • National consultants • Train 21 sites (not MTs) • Rich Toscano • Mary Sperrazza (local)

5. Good Day Assumptions • MTs from good sites will provide good training (corollary: MTs will do what Rich does) • Fidelity will be used as a QI tool • MTs will learn fidelity assessment • Programs will benefit from telephone supervision

6. Bad Day Assumption Impossible, unfeasible plan doomed for failure

7. What happened?

8. Activities: 21 MT Sites • Kick-off meetings • Louisville - September 2004 • Palo Alto - March 2006 • most between June – November 2005 • Fidelity every six months • All sites (no refusals) • Complete data for 2 years

9. MT Fidelity: November 20089 of 21 sites at full implementation Consistent with SE Fair SE Implementation Not SE 52.2 60.8 64.6 64.2 64.8 63.5 n = 20 n = 17

10. Any Competitive Work at 21 MT sites (last 90 days) n = 456 M = 45 SD = 24 n = 563 M = 40 SD = 25 n = 745 M = 46 SD = 22 n = 45 M = 39 SD = 22 n = 241 M = 41 SD = 29 number of days worked

11. What did we learn?

12. Not an impossible, unfeasible plan doomed for failure • Some MT sites are implementing SE well • 9 sites at full implementation • Mean fidelity score around 65 • Veterans are getting jobs • Some sites hiring extra CWT/SE staff • but… faulty assumptions

13. 1. Good sites will provide good training • Implementation and training are different skill sets • Giving feedback • Earning respect • Assessment • Good MTs have good supervisors • Role models? • Hire better people?

14. 2. Fidelity will be used as a QI tool3. MTs will learn fidelity • Fear factor • Teaching to the score • Where are the principles? • Implementation, training AND fidelity assessment are all different skill sets • Asking open-ended questions • Assessing not teaching • Content • Unrealistic expectations?

15. 4. Sites will use telephone supervision (3 years) number of calls site

16. What now?: Phase 2 • Train the MTs • West Haven • New application of MI (Instructional Interviewing?) • How to teach SE principles • Focus on positives • With Rich • Fidelity • Technical assistance • What leads to sustainability?