Promoting Quality Prevention Counseling Project: What have we learned?

1. Promoting Quality Prevention Counseling Project: What have we learned? Spring 2005 Texas Tour Dallas, Fort Worth, Houston, Midland, Austin

2. Agenda • Welcome and Introduction • Background • Overview of project • Implementation experiences by sites • General evaluation findings • Next Steps • Q & A

3. Background • Revised HIV Counseling, Testing and Referral (CTR) Guidelines, November 2001

4. Background Cont’d • CDC’s Project RESPECT: Evidence-based intervention showing significant reduction of STDs with protocol-based HIV prevention counseling • RESPECT-2: Refined HIV prevention counseling protocol & further developed “counseling quality assurance” methods

5. RESPECT Methodology • 5758 heterosexual, HIV-negative patients older than 14 years who came in for STD examination • Five public STD clinics (Baltimore, Denver, Long Beach, Newark and San Francisco)

6. Project RESPECT Results*: HIV Prevention Counseling Effective (*p<0.05) Kamb, M.L., et al (1998) Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases, JAMA, 280 (13):1161-1167

7. How could we translate this intervention into a real-world setting? • Risk Reduction Specialist support • Supervisor support • Practical tools

8. Goals of the Project • Develop and evaluate tools to support protocol-driven prevention counseling based on the RESPECT model • Develop and evaluate QA procedures • Better understand the barriers and facilitators of good prevention counseling

9. Definitions • Evidence-based interventions • Interventions that have demonstrated desired outcomes through rigorous research • Core elements • Components of the intervention that are believed to be essential to achieve the desired behavior change • Protocol • A structured approach to achieve core elements

10. Definitions • Client-centered prevention counseling • One-on-one interactions with risk-reduction as its primary goal • Risk Reduction Specialist • A trained specialist responsible for maintaining the focus on a client’s specific risk reduction needs • Tools • Job aides to ensure fidelity to core elements of protocol

11. Site Locations Tarrant County Health Dept. Ft. Worth, TX Resource Center Of Dallas Dallas, TX cc Valley AIDS Council, McAllen, TX Brownsville, TX cc City of Laredo Health Dept. Laredo, TX

12. What was introduced during the project? • Counseling protocol • Training on the protocol • Counseling tools • Spiral book with goals and sample questions • Laminate “wheel” • Documentation form with space for RR plan and referrals • Personal review form

15. Session Documentation Form

16. What QA activities were part of the project? • QA protocol • Emphasizing standardized preceptorship, observation, routine meetings, documentation review, and feedback on observations and documentation • QA tools • Supervisor observation tool • RRS self assessment • Chart abstraction and summary tools

17. Quality Assurance Tools

18. Comparison of Core Elements

19. Implementation • Training developed for protocol, tools and QA • Supervisors and Risk Reduction Specialists trained in October 2003 • On-site and off-site TA provided for start-up • Staggered and tailored implementation of protocol, tools, and QA • Additional ongoing TA after start-up

20. What Do We Want to Learn? • Can you implement protocol-based prevention counseling with existing resources? • Did the protocols and tools help them implement the intervention with fidelity? • What were some of the facilitators and barriers of the implementation of the protocols?

21. Evaluation Design • Two data collection phases: Pre- and Post-intervention • Evaluation data triangulation: 9 complementary quantitative and qualitative instruments

22. Quantitative Instruments • Supervisor time logs (pre and post) • Client Questionnaires (pre and post) • Counseling chart reviews (pre and post) • Observations of counseling by evaluators (pre and post)

23. Qualitative Instruments • Risk Reduction Specialist (pre and post) • Supervisors (pre and post) • Site Program Managers (post only) • WAP (post only) • TDH (post only)

24. Results Lessons Learned

25. Did the protocols and tools help implement the intervention with fidelity?

26. Observations: Initial SessionGoals *P<.05

27. Observations: Follow-up SessionGoals *P<.05

28. Client survey: Initial sessionGoals *P<.05

29. Client survey: Initial sessionGoals (cont’d) *P<.05

30. Client survey: Follow-up sessionGoals *P<.05

31. Client Surveys: Client Participation *P<.05

32. Chart reviewsGoals *P<.05

33. Significant changes seen in initial sessions

34. Significant changes seen in follow up sessions

35. “That you had to follow every single task even though they didn't all apply to everybody [is a problem]. Protocol doesn't allow for individual counseling styles or use of skills RRS's have received at prior trainings…it seems cumbersome and redundant to use this protocol with clients with very few risks - although it's easy enough to move through the protocol by saying this doesn't really apply to you [for certain tasks].” -Risk Reduction Counselor

36. “[The protocol improved the quality of my counseling] because I had a structure to make sure I wasn’t leaving anything out.” -Risk Reduction Counselor

37. Observations: Initial Visit Use of Counseling Skills *P<.05

38. Observations: Follow-up Session Use of Counseling Skills *P<.05

39. Client Surveys: Client experience *P<.05

40. What were some of the facilitators and barriers of the implementation of the protocols?

41. Overall themes and feedback • Delivery of protocol-driven prevention • Provided structure • Improves with practice • Aided in identifying risk behaviors and patterns • Protocol questions felt rigid • More training and TA is essential

42. Overall themes and feedback (cont’d) • Spanish version tools are needed • Supervisor buy-in is essential • Supervisor’s other responsibilities need to be considered due to time constraints of quality assurance • Difficult with certain clients (such as low risk, outreach, drug treatment and jail)

43. Using the Counseling Tools • Most of the RRS found the cards to be the most helpful of the tools (72%) • Cards help ensure you cover everything in order (44%) • Wheel was not as helpful (83%) • Spanish version of the tools is needed

44. Counseling QA by RRS • Most helpful • Observation by supervisor • General feedback • Role play • Observation by peer/peer observation/document review • Case conference

45. Counseling QA by RRS (cont’d) • Least helpful • Documentation review • Observation by peer • Case conference/self assessment

46. Counseling QA by Supervisor • Most helpful • Observation by supervisor • General feedback • Document review • Case conferences • Least helpful • Observation by peer • Self-assessment

47. QA Activities by RRS

48. “[The supervisor observation form] is better because it is less subjective and more structured. …the priorities of the tasks are made clear by the forms and that feedback using these forms makes the whole process self-reinforcing [the process of understanding the expectations of the protocol, using the protocol, and getting feedback—all have the same language, structure, and expectations]…the new feedback is less stressful for everybody, including the observer for the stated reasons.” ---Risk Reduction Specialist

49. I think this new protocol is great—fabulous! Before when they first told us about the program and we went to training, we were all ‘iffy’ and said ‘it’s not gonna work’ ‘no way in heck’ it would be accepted by the people. Now that we are implementing it, we are doing a great job. When you have to write steps, the clients leave with RR plan in hand, a referral, an appointment card with the date on it in hand. As for review forms used by the supervisor on documentation, etc. You have the form yourself to be able to discuss ‘met’ or ‘not met.’ ---Risk Reduction Specialist

50. What has been done? • Changes to training • Trainers have bought in • Preceptorship is done first, then attend training • More time for role play • Develop their own questions for each step • Not a gripe session • Sites learning from each other • Role plays/Peer observations for practice • Sign in waiting room for length of sessions • Regularly scheduled QA sessions