Assessing Student Competencies: A Model from Physical Therapy

1. Assessing Student Competencies: A Model from Physical Therapy Lynn Fitzgerald, MEd, PT Director of Clinical Education Department of Physical Therapy

2. OVERVIEW • Standards for American physical therapy programs • The University of Pittsburgh Physical Therapy program • The University of Pittsburgh Clinical Internship Evaluation Tool • A physical therapy Clinical Instructor training program

3. Inconsistency in Curricular Models Across Programs

4. Variability in Clinical Education Hours Across Programs

5. APTA Clinical Performance Instrument Benefits • Standardized • Validated • Web based system available • Online training for clinical instructors Problems/Issues • Too long (23 pages) • Expense • CIs don’t complete training • Items not relevant to all settings/experiences • Scoring criteria complicated Task Force on the Development of Student Clinical Performance Instruments. The development and testing of the APTA Clinical Performance Instruments. Phys Ther. 2002; 82: 329-353.

6. The University of Pittsburgh Doctor of Physical Therapy Program • Traditional model with problem based learning and flipped classrooms used as teaching tools • 130 weeks long with 123 credit hours (34 in clinical education) • 54 weeks of full time and 42 weeks part time for total of 2664 hours of clinical education

7. Multiples Methods of Assessing Student Behaviors and Skills

8. Rationale for Choosing an Alternate Clinical Evaluation Tool Too long Anchor too low for a yearlong clinical Disagreed with definition of entry level competencies

9. Development of the University of Pittsburgh Clinical Internship Evaluation Tool (CIET) Fitzgerald LM, Delitto A, Irrgang JJ. Validation of the Clinical Internship Evaluation Tool. Physical Therapy.2007;87:844-860. • Defined what we wanted our graduates to look like • Identified the behaviors and skills a PT should demonstrate • Defined our standard for achievement

10. Our students are evaluated against a “competent clinician who is able to skillfully manage patients in an efficient manner to achieve an effective outcome”.

11. Fitzgerald LM, Delitto A, Irrgang JJ. Validation of the Clinical Internship Evaluation Tool. Physical Therapy.2007;87:844-860. • Items on the CIET are representative of the skills and behaviors important for physical therapists to achieve • The CIET has sufficient internal consistency • The CIET appears to measure the clinical competency level of the student • The CIET allows the CI to grade the student above “entry-level” for the longer clinicals • The CIET is short, clinician is able to complete in reasonable time

12. Changes/Additions to the Tool Over Time • Changed Likert scale to improve grading for patient management Initial Scale Revised Scale

13. Changes PATIENT / CLIENT MANAGEMENT SKILLS EXAMINATION Week 1:  Accurately takes vital signs of patients.  Accurately performs an assessment of patient’s ROM. Week 2:  Accurately performs manual muscle testing.  Accurately assesses patient’s bed mobility.  Obtains an accurate history from a patient chart/patient/family.  Completes an orientation assessment of patient. Week 3:  Completes a basic gait assessment.  Completes a sensory examination.  Completes an assessment of patient’s sitting and standing balance • Delineated expectations for each clinical through development of Clinical Performance Benchmarks

14. Changes • Training of CIs through inservices and online training • Students use tool for self-assessment

15. The APTA Clinical Instructor Education and Credentialing Program 15 hours of education Assessment Center Credentialed CI APTA Clinical Instructor Education & Credentialing Program (CIECP). American Physical Therapy Association Website

16. The Basic CI Credentialing Course. • The Clinician as a Clinical Educator • Readiness to Learn • Facilitating Learning in the Clinical Environment • Performance Evaluation in the Clinical Environment • Legal, Regulatory, and ADA issues in Clinical Education • Managing the Exceptional Student APTA Clinical Instructor Education & Credentialing Program (CIECP). American Physical Therapy Association Website

17. Content APTA Clinical Instructor Education & Credentialing Program (CIECP). American Physical Therapy Association Website. • Preparing & planning for a clinical experience • Development of a learning experience for the student • Use of effective supervision skills during the experience including questioning and feedback • Development of performance evaluation skills • Identification and management of the student in an exceptional situation • Understanding of the legal implications for clinical education

18. Clinical Instructors Value Training • Increased knowledge of legal responsibilities • Can structure a more organized learning experience • Learned how to address specific problem situations • Evaluate more objectively • Provide more constructive feedback • Promote adult learning in their students • Increased confidence in their skills Buccieri KM, Schultze K, Dungey J, Kolodzief T, Malta S,Marocco S, et al. Self-reported characteristics of physical therapy clinical instructors: a comparison to the APTA guidelines and self assessments for clinical education. J PhysTher Educ. 2006; 20(1): 47-55. Kettenbach V. The role of participating in a clinical instructor training curriculum in preparing clinical instructors to comply with the APTA Guidelines for Clinical Instructors (doctoral dissertation). St. Louis, MO: Saint Louis University 1999. Wilson M. Clinical educators: convinced about credentialing. PT Mag. 2002; 10: 42-46.

19. Studies Reporting Impact on Student’s Experience. • Housel & Gandy (2008) showed that while student overall performance was not significantly different, CI credentialing had a significant interaction with the level of improvement students achieved from midterm to final • Housel & Gandy (2010) found students rank credentialed CI’s as more effective overall then non-credentialed CIs • Studies are limited and there needs to be more research • Housel N & Gandy J. Clinical Instructor Credentialing and its effect on student clinical performance outcomes. J PhysTher Educ. 2008; 22 (3): 43-51. • Housel N, Gandy J, & Edmondson D. Clinical Instructor Credentialing and Student Assessment of Clinical Instructor Effectiveness. J PhysTher Educ. 2010; 24 (2): 26-34.

20. Suggestions for Development and Implementation of a Clinical Instructor Training Course • Much of the material can be presented in an online format • Learners now want LOTS of digital scenarios (you tube) • Learners want to hear more about how to “do” it and how to address problem situations, not just dry information • Keep assessment short, could definitely be online • Information can also be presented in entry level curriculum as part of teaching/learning principles needed for patient care • CI’s report they are more effective post training

21. In Summary • Physical Therapy programs are extremely variable in how they teach, evaluate, and the length of clinical education • A clinical performance evaluation tool must assess the specific behaviors and skills that you want your graduates to demonstrate with a defined outcome. • An assessment tool only works if it is used in the same way it has been validated so training of your clinical instructors is key • Clinical Instructor training can be done in combination during entry level training, through online courses, and “short” continuing education courses

22. References American Physical Therapy Association Web Site. Voluntary Training and Credentialing Program for Clinical Education. Buccieri KM, Schultze K, Dungey J, Kolodzief T, Malta S,Marocco S, et al. Self-reported characteristics of physical therapy clinical instructors: a comparison to the APTA guidelines and self assessments for clinical education. J Phys Ther Educ. 2006; 20(1): 47-55. Commission on Accreditation in Physical Therapy Education website. English L, Wurth RO, Ponsler, M, & Milam A. Use of the physical therapist clinical performance instrument as a grading tool as reported by academic coordinators of clinical education. J Phys Ther Educ. 2004; 18 (1): 87-92. Fitzgerald LM, Delitto A, Irrgang JJ. Validation of the Clinical Internship Evaluation Tool. Physical Therapy.2007;87:844-860. Housel N & Gandy J. Clinical Instructor Credentialing and its effect on student clinical performance outcomes. J Phys Ther Educ. 2008; 22 (3): 43-51. Housel N, Gandy J, & Edmondson D. Clinical Instructor Credentialing and Student Assessment of Clinical Instructor Effectiveness. J Phys Ther Educ. 2010; 24 (2): 26-34. Kettenbach V. The role of participating in a clinical instructor training curriculum in preparing clinical instructors to comply with the APTA Guidelines for Clinical Instructors (doctoral dissertation). St. Louis, MO: Saint Louis University 1999. Task Force on the Development of Student Clinical Performance Instruments. The development and testing of the APTA Clinical Performance Instruments. Phys Ther. 2002; 82: 329-353 Wilson M. Clinical educators: convinced about credentialing. PT Mag. 2002; 10: 42-46.