Assessment of Trainee Competencies: Preliminary Psychometric Study

1. Assessment of Trainee Competencies: Preliminary Psychometric Study David J. Martin, Ph.D. Harbor-UCLA Medical Center Geffen School of Medicine at UCLA

2. Models of Competency • Cube Model of Competency Development (Rodolfa, Bent, Eisman, Nelson, Rehm & Ritchie, 2005) • Functional Competency • Foundational Competency • Stages of Professional Development

3. Models of Competency • Cube Model (Rodolfa et al., 2005): • Functional Competency • Assessment-Diagnosis-Case Conceptualization • Intervention • Consultation • Research Evaluation • Supervision-Teaching • Management-Administration

4. Models of Competency • Cube Model (Rodolfa et al., 2005): • Foundational Competency • Reflective Practice-Self Assessment • Scientific Knowledge-Methods • Relationships • Ethical-Legal Standards—policy • Individual-Cultural Diversity • Interdisciplinary Systems

5. Models of Competency • Cube Model (Rodolfa et al): • Stages of Professional Development • Doctoral Education • Doctoral Internship/Residency • Post-doctoral Supervision • Residency/Fellowship • Continuing Competency

6. Models of Competency

7. Models of Competency • Practicum Competencies Outline • Association of Directors of Psychology Training Clinics (ADPTC) Practicum Competencies Workgroup (February 27, 2004) • Council of Chairs of Training Councils (CCTC) Practicum Competencies Workgroup (March 25, 2004) • Final CCTC comments, November, 2004 • Summarized by Hatcher & Lassiter (2006)

8. Models of Competency • Practicum Competencies Outline • Levels of Competence • Novice • Intermediate • Advanced • Proficient • Expert

9. Models of Competency • Practicum Competencies Outline • Areas of Competence • Relationship/Interpersonal Skills • With Patients/Clients/Families • With Colleagues • With Supervisors • With Support Staff • With Teams at Clinic • With Community Professionals • For the Practicum Site Itself

10. Models of Competency • Practicum Competencies Outline • Areas of Competence • Skills in Application of Research • Intervention Skills • Psychological Assessment Skills • Consultation Skills/Interdisciplinary • Collaborations • Diversity (Individual & Cultural Differences)

11. Models of Competency • Practicum Competencies Outline • Areas of Competence • Ethics • Professional Development • Practical Skills to Maintain Effective Clinical Practice • Professional Development Competencies • Metaknowledge/Metacompetencies • Development of Leadership Skills • Supervisory Skills

12. Harbor-UCLA Assessment • Psychological assessment skills

13. Harbor-UCLA Assessment

14. Warning ! Attempts at complete review of the following slides may result in serious information overload!

15. Internal Consistency

16. Internal Consistency

17. Descriptive Statistics

18. Descriptive Statistics

19. Descriptive Statistics

20. Descriptive Statistics

21. Inter-Scale Correlations • Fellows’ Inter-Scale Correlations Ranged From .39 (Relationship Skills with Support Staff – Leadership Skills) - .93 (Assessment Skills – Consultation Skills) • Externs’ Inter-Scale Correlations Ranged From .63 (Relationship Skills with Team – Ethics) - .95 (Intervention Skills – Consultation Skills)

22. Pre – Post Comparisons

23. Pre – Post Comparisons

24. Pre – Post Comparisons

25. Pre – Post Comparisons

26. Discussion • Preliminary data suggest that competencies can be measured reliably • Inter-scale correlations raise important questions concerning competency constructs

27. Discussion • Future Directions • Need for additional data to inform competency evaluation process • More controlled study of competency scales (e.g., mid-term and final ratings) • Factor-analytic approaches may address questions of inter-scale shared variance • Need for validation strategies