Supervision Strategies to Enhance Implementation and Fidelity to EBP

1. Supervision Strategies to Enhance Implementation and Fidelity to EBP Kelly Pitocco, LISW-S, LICDC University of Cincinnati Corrections Institute kelly.pitocco@uc.edu

2. State Policy Putting Supervision in Context of Implementation Process Funder To Do . . . Practitioner Training

3. Supervisor Training and Support

4. Supervisor Training and Support

5. The Four Phases of Learning Preparation Often Skipped Presentation Practice Performance

6. Preparation Assessing and Readying for Change

7. Preparation • Get them interested • Encourage positive feelings about new learning • Create Learning Environment

8. Survey of AOD Professionals Past Year use of CBT MET MI 12 Step Facilitation 83% 75% Reported Currently Using EBPs

9. However, Many have Negative View of Curriculum-based Treatment

10. Attitudes About EBP Most Clinicians either Believe they are Using EBP Or Want to Use EBP

11. Positive Attitudes • Using manual help counselor evaluate and improve skills • Treatment manual will enhance outcomes • Counselors are ethically obliged to use EBP

12. Negative Attitudes • EBP make staff more like technician than caring people • Treatment manuals appropriate for research but not real life

13. Readiness Factor

14. Prepare for Implementation • Determine if the organization and program are ready to adopt model • Assess staff readiness to change • Set up a multidisciplinary change committee • Program leadership and change committee develop the training and follow-up plan

15. Strategic Plan Readiness Assessment Project Management Change Management Staff Training Staff Supervision Rating Coaching Monitoring Fidelity CQI Evaluation Areas for Readiness • Staff Training • Staff Supervision • Rating • Coaching

16. Goals for Preparation • Get staff out of a passive or resistant mental state • Remove learning barriers • Arouse interest • Give staff positive feelings about, and an incentive to learn • Create active learners • Establish a learning group

17. Techniques • Provide positive suggestion • Discuss benefits to clinician and client • Set clear, meaningful goals • Raise curiosity • Create a safe and positive environment • Calm staff fears • Identify and remove barriers to learning

18. Techniques • Raising questions and posing problems • Getting staff involved in implementation

19. Learner Barriers I don’t have Time Need to Save Face This is the Way We’ve Always Done it If it Ain’t Broken, Don’t Fix it NO WIIFM The way I do it Is Good Enough Personal Issues I already do this

20. Learner Benefits • Brainstorm benefits

21. Curiosity Arousal • Give people problems to solve in teams • Send staff on fact-finding missions • Play question and answer games • Self-discovery activities

22. Presentation

23. Broader than Training • Need a mechanism to accomplish: • Acceptance of change • Means to incorporate change • Reinforced at all levels of system

24. Typically underestimate the time and effort needed to: • Train • Implement • Achieve fidelity to the model

25. Occasional Quotes from Trainees “I was told I have to be at this training. I have no idea why I am here.” “That won’t work at my Site/with our clients/within the time we have.” “I’m just here to get my 30 hours/CE’s/ mandatories done”

26. Training Culture Successful training experience wanes with each disengaged participant

27. Typical Reaction of Trainees • Excited to learn a new clinical intervention • Enthusiastic and committed to trying it with clients • Although there wasn't much time for skill-building during the session, they have their notes and want to try it out

28. Reality Returns • Then they get back to the site • to the routine • to the caseload • to the demands by courts • to UR or required contacts • to the supervisor and co-workers who didn't take the training • to the clients who aren't prepared for something new or different

29. Not Prepared • Notes now seem incomplete • Can’t recall details from the training • Enthusiasm and new knowledge begin to fade Adopting new practices in the context of everyday work is difficult and frustrating

30. Do One-Shot Trainings Work? • 15 hour training on MI • Pre-training baseline audiotape • Helpful Responses Questionnaire

32. Self-Report • Participants over-inflated their skills of using MI after the training • Use of MI declined with time following the training (about 50%) • Skills were about ¼ of proficient

33. Maximizing Gains from Classroom Training • Use of knowledge-based pre/post tests • Use of knowledge-based proficiency tests • Use of skill-based rating upon completion of training • Mechanism for use of data • Rated competent or continued development until reach competent

34. Role of Trainer • Training is a means to an end – not an end in itself • Trainer is Performance Consultant • Trainer partners with the learner • Link to business need • Proactive and reactive • Front end assessment and evaluation of performance

35. Evaluation • Knowledge Test – pre/post • Competency – skill-building sessions • Skill check off • Structure through policy and procedure • Program Integrity Evaluation • CQI or outcome evaluation

36. Supervisory Strategies for Enhancing Training Transfer • Communicate expectations prior to training • Demonstrate involvement of training content – integrated (clinical and staff meetings, paperwork, etc) • Hold learner accountable for applying content in work (rewards/sanctions)

37. Supervisory Strategies for Enhancing Training Transfer • Demonstrate and Model the skills • Provide learning and practice opportunities • Integrate learning objectives into performance appraisal • Observe and provide feedback and coaching • Booster training sessions

38. What would the Supervisor need to be able to do those items? Small Group Discussion

39. Remember Two Part Process Change Management And Skill Development

40. Areas to Address • Skill Deficit • Resistance • Both

41. Role of Clinical Supervisor • Safety • Quality • Effectiveness • Compliance Oh yeah, and . . . . Oversee services Administrative Tasks Daily Crisis Management Prepare Reports Manage Caseload Provide coverage Hire Staff All other duties no one else wants to do

42. Evidence Supporting Supervision • Conditions • Manual Only • 14 hour Workshop • Workshop + Feedback • Workshop + Coaching • Workshop + Feedback + Coaching All had initial Skill Acquisition Miller, et. al., A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing. Journal of Consulting and Clinical Psychology (2004)

43. Four Months Later • Conditions • Manual Only • 14 hour Workshop • Workshop + Feedback • Workshop + Coaching • Workshop + Feedback + Coaching Could not Detect Who had Training Only condition that maintained benefit after 4 Months

44. CBT Study • Conditions • Manual Only • Manual + Web-based Training (40 hours) • Manual + Training + Supervision (observation and feedback) Sholomskas, et. al., We don’t Train in Vain: Three Strategies of Training Clinicians in CBT . Journal of Consulting and Clinical Psychology (2005)

45. CBT Study • Conditions • Manual Only – No Transfer • Manual + Web-based Training (40 hours) - Modest Transfer • Manual + Training + Supervision (observation and feedback) - Proficient

46. Sholomskas, et. al., 2005 “Face to face training followed by supervision may be essential for effective technology transfer and raises questions about whether practitioners should feel competent to administer an empirically-supported treatment on the basis of reading a manual alone.”

47. Current State • What is your current assessment of staff in providing evidence-based practices? • What are the challenges in achieving staff proficiency? • What changes could facilitate improvement in clinical supervision?

48. Observed Contacts • On a scale of 1 - 10 • How important is it for the clinical supervisor to have direct observation to effectively provide supervision? • If you rated high – why? • If you rated low – why not? What do you think your staff would say?

49. Behind Closed Doors Ever Make you Nervous???

50. Types of Interventions Used Score of 4 Considered Proficient