Identifying a Path Toward Measuring Integrated Team-Based Core Competencies

1. Identifying a Path Toward Measuring Integrated Team-Based Core Competencies C.R. Macchi, PhD, Clinical Associate Professor, Arizona State University Patti Robinson, PhD, Director of Training and Program Evaluation, Mountainview Consulting Group Rodger Kessler, PhD, Research Associate Professor, Arizona State University Session # I6 CFHA 18th Annual Conference October 13-15, 2016  Charlotte, NC U.S.A.

2. Faculty Disclosure C.R. Macchi and Rodger Kessler have NOT had any relevant financial relationships during the past 12 months. Patti Robinson is a consultant for Mountainview and received royalties as an author for one of the referenced titles during the past 12 months.

3. Learning ObjectivesAt the conclusion of this session, the participant will be able to: • Participants will review existing descriptions of behavioral health core competencies needed to effectively perform in primary care. • Participants will explore overlapping competencies required of each profession on a care team • Participants will investigate potential approaches needed to evaluate core competencies • Participants will consider components needed to develop valid, reliable measurement tools of core competencies.

4. Bibliography / References Miller, B., Gilchrist, E., Ross, K., Wong, S., Blount, S., Peek, C.J. (2016, February). Core competencies for behavioral health providers working in primary care. Prepared from the Colorado Consensus Conference. Interprofessional Education Collaborative Expert Panel. (2011). Core competencies for interprofessional collaborative practice: Report of an expert panel. Retrieved from Washington, D.C.: https://ipecollaborative.org/uploads/IPEC-Core-Competencies.pdf Kinman, C., Gilchrist, E., Payne-Murphy, J., & Miller, B. (2015). Provider- and practice-level competencies for integrated behavioral health in primary care: A literature review (14-0073-EF). Retrieved from Rockville, MD: McDaniel, S. H., Grus, C. L., Cubic, B. A., Hunter, C. L., Kearney, L. K., Schuman, C. C., . . . Johnson, S. B. (2014). Competencies for psychology practice in primary care. American Psychologist, 69(4), 409-429. doi:10.1037/a0036072 Miller, B., Gilchrist, E., Ross, K., Wong, S. G., Blount, A., & Peek, C. J. (2016). Core competencies for behavioral health providers working in primary care. Retrieved from http://farleyhealthpolicycenter.org/wp-content/uploads/2016/02/Core-Competencies-for-Behavioral-Health-Providers-Working-in-Primary-Care.pdf Robinson, P. J. & Reiter, J. T. (2015). Behavioral consultation and primary care: A guide to integrating services, 2nd Edition. NY: Springer.

5. Learning Assessment • A learning assessment is required for CE credit. • A question and answer period will be conducted at the end of this presentation.

6. Core Questions • What are the core domains, how do you develop your competencies? • Who assesses the findings and how are the competencies related to program outcomes? • How do you make it sustainable? • What are the next steps toward developing a valid, reliable measure of BH competence for each team member? • What are the central themes and items that could be used with multiple members of an integrated team?

7. Doctoral Training Approach to Evaluation Pre-professional Internship Evaluation

8. Core Competencies & Skill Development McDaniel, Campbell, Hepworth & Lorenz (2005) Robinson & Reiter (2015) McDaniel, Doherty & Hepworth (2014)

9. Tool Development Purpose of the tool • Provide a way for student interns and supervising raters to focus evaluation on key integrated, team-based practices in primary care settings • Student behavioral demonstrations of those practices • Student engagement with patients and other team members • Develop crosswalk of clusters/domains – knowledge, skills, and attitudes • Competencies for Psychology Practice in PC - Interorganizational Workgroup (McDaniel et al., 2014) - six competency clusters • Core competencies for integrated behavioral health and PC (SAMHSA, 2014) • Behavioral benchmarks for each competency • Provider-Level Competencies for Integrated Behavioral Health in PC (AHRQ, 2015) • Core Competencies for Behavioral Health Providers Working in Primary Care (Farley Health Policy Center, 2016) • Behavioral benchmarks for each competency • Develop common clusters • Student performance standards for each domain reflect the skillset that is assumed to be • appropriate in the primary care environment • applicable to clinics of varied levels of integration • associated with improved care delivery • associated with increased levels of integrated behavioral healthcare delivery

10. Tool Development • Develop items that operationalize clusters based upon measureable behavioral anchors • Items of each evaluation related to the rater’s role and observations of BHCs clinical practice • Convergent items by domain • Discriminant items between domains • Align anchors based upon the three worlds – clinical, operational, & financial systems (Peek, 2008) • Develop evaluation domains that align performance between systems • Develop standard scale for rating each item • Facilitate developmental assessments of each domain • Identify student intern progress over the full internship course(s) • Make the full points for each domain meaningful and reflective of excellent, independent performance

11. Tool Development • Field leader reviews and comments and comparison with available rating tools • Rodger Kessler • DBH faculty Compared tool with existing tools • TRI-SERVICE IBHC CORE COMPETENCY TOOL • Benchmark behaviors • 2-point rating scale (pass/fail) • PRIMARY CARE BEHAVIORAL HEALTH PROVIDER ADHERENCE QUESTIONNAIRE (PPAQ) • 5-point rating scale (never to always) • BEHAVIORAL HEALTH CONSULTANT CORE COMPETENCY SCALE • 5-point rating scale (low to high)

12. Elements of the Tool • Categorized core competencies • Three worlds – clinical, operational, financial • Behavioral anchors • Behavioral settings • Developmental scale

13. Core Competency Domains and Skills • Conceptual and professional development • Systems orientation • Primary care culture and adaptation • Professionalism • Practice-based learning • Biopsychosocial orientation • Quality improvement • Clinical skills/practice • Screening and assessment of BH issues related to medical conditions • Population-based approaches • Evidence-based interventions • Informatics and data

14. Core Competency Domains and Skills • Practice management • Documentation • Time management • Resource management • Compliance with clinic policies and protocols • Collaboration • Communication with providers and patients • Interprofessional relationships • Care coordination • Identifies and supports medical team shared goals • Support patient engagement on care team • Finances and utilization of resources – demonstrates an awareness and responsiveness to the financial implications of clinical and operational activities

15. Sample Items

16. Use of the Evaluation Tool • Multiple raters • On-site – site liaison and medical preceptor • Program faculty • Student intern self-assessment • Program faculty conducts continuous reviews • Weekly meetings • Mid-semester review comparing ratings • End-of-semester, comprehensive review comparing ratings • Program faculty meets with student intern to identify areas of progress and growth • Course grade is based upon • Student attainment of baseline skills • Demonstration of progress across core competency domains • Student meets thresholds for activity metrics

17. Companion Activity Metrics • Measuring student intern activities (targets) • Direct patient contact hours • % of total hours direct patient contact (40%) • Average time with each patient (less than 30 minutes) • % of patients seen who were administered approved, validated outcome measures with patients (65%) • Average number of outcome measures administered to each initial and follow-up patient visit (1 measure) • % of patients seen who were provided a self-management resource (25%) • Administrative hours • Training and education (e.g., IBH presentations, orientation) • Interprofessional interactions and team-building (e.g., huddles, warm handoffs, team meetings) • Patient care planning (e.g., case consultations, reviewing medical records, preparing for patient groups) • Data management (e.g., collecting/analyzing patient data, entering medical reports)

18. In-Field Retraining Approach to Evaluation Behavioral Health Consultant Core Competency Tool

19. Development of PCBH BHC Tools • 1996: Initial Primary Care Behavioral Health (PCBH) Core Competency Tool for Behavioral Health Consultants (BHCs) ((Refined 2002, 2007, 2015) • Other PCBH CC Tools: BHC Assistant CC Tool, BHC Mentor CC Tool, Clinic Leadership CC Tool • Additional Tools Related to BHC Tool • Referral Barriers Questionnaire 1 • IPCMH Integration Tool 2

20. PCBH BHC Tool Development • Purpose of tools • Inform BHC training • Inform training of PCPs, RN, other staff • Prepare Clinic Leadership team to provide essential preparation and on-going evaluation and management • Evaluate job performance on an annual basis • Inform performance improvement planning • Influence graduate training (BH & PC) • Process for CC Tool development • Analysis of competencies associated with anticipated improvements to population health, patient and provider satisfaction and value of services, identifying and addressing barriers to achievement of outcomes

21. Core Competency Domains and Skills • Clinical • Consultation • Practice Management • Team-based Care • Documentation • Administrative

22. Training Venues • BHC • Phase 1 (foundation skills) • Group based • Didactic, modeling, guided rehearsal • Phase 2 (on-the-job skills) • In-clinic in process of delivering care • Simultaneous effort to train Leadership Team, PCPs, and Nursing Staff in PCBH Competencies • On-going monitoring and feedback • Fidelity metrics • PCBH program outcomes • PCP & BHC performance BHC Mentor Training

23. ZZZ Clinic: PCP Satisfaction with IBHC Services (n = 11) (1=no apparent benefit, not helpful to 10 = extremely helpful)

24. ZZZ Clinic Integration Tool Results

25. Validation of Competent PCBH Program Provide comparison among clinics; distribute Monthly to all PC providers, teams, management

26. Evolving PCBH • Re-vamping discipline specific training programs to train students for the needed workforce • Inter-disciplinary training venues • PCBH faculty in Family Practice residencies • University of Texas R – Deepu George, Leslie Allison • Central Washington Family Medicine – Bridget Beachy, David Bauman, Kirk Strosahl

27. Evolve PCBH • Supplementary CC Tools • Focused Acceptance and Commitment Therapy Competency Self Assessment (FACT-CAT) • Brief Intervention Competency Assessment Tool • (BI-CAT) • Fidelity • On-going measurement of fidelity to practice • Link of trained competencies, practiced with fidelity to optimal health care outcomes • Training Mentors • Mentor CC Tool

28. Assist PCBH Expansion • To assist you in your pursuit of competence . . . • m PCBH Competency Tools* Supplementary Competency Tools Other PCBH Tools www.MtnviewConsulting.co *Robinson & Reiter, 2015; See www.behavioralconsultationandprimarycare.com

29. Session Evaluation Please complete and return the evaluation form before leaving this session. Thank you!