Authentic Performance Assessment

1. Authentic Performance Assessment Allison D.Spinelli,D.O. Assistant Professor of Anesthesiology Residency Program Director

2. Authentic Performance Assessment • Background • What is competency • How to assess competency • Examples of APA model • Competency rating scale • Pros/cons • Looking ahead

3. Background • Why change evaluation system • Standardizes faculty approach to evaluating residents • Helps to define resident’s competency level for a particular year/rotation • Better defines competency

4. What is Competency • ‘Know it when I see it”

5. How to Assess Competency • Authentic Performance Assessment Model

6. Authentic Performance Assessment Model • Real-time assessment of resident performance during actual patient care • Clearly defines the skills/knowledge the resident must possess to be deemed competent at a task • Specific to resident year of training • Specific to rotations

7. Examples of APA Model • CAY 1 Mentorship period • Occurs 2 weeks after start of 1st year • Resident should be able to set-up room,perform machine check,start iv,intubate uncomplicated airway

8. Examples of APA Model • Four month APA for CAY1 resident • All of previous skills/knowledge plus: • Induction of GA

9. Examples of APA Model • 8 month APA for CAY1 resident • All of previous skills/knowledge plus • Maintenance of GA and intra-op differential dx of vital sign abnormalities

10. Examples of APA Model • 12 month APA for CAY1 resident • All of previous knowledge/skills as well as emergence of patient from GA and post-op care

11. Examples of APA Model • CAY 2 resident on cardiac anesthesia rotation • Pre-op,line placements,induction,airway • Manage prebypass period • Manage postbypass period • Manage transfer of care to CSICU • Diagnose and treat postop complications

12. Examples of APA Model • CAY 3 residents on any rotation and for the most acute and complex patients • Pre-op assessment and anesthetic plan • Line placement • Induction and airway management • Intra-op management of data • Emergene and transfer of care • Post-op complications

13. Competency Rating Scale • Rubric system • Highly competent • Competent • Becoming competent • Not competent • Gauges the degree of resident’s ability to perform independently vs. need for attending intervention

14. Examples of Competency Rating Scale • Highly competent • The resident is smooth and efficient • The resident adapts knowledge,judgement,and skill to start the case and induce anesthesia with no significant errors and most appropriately • The resident does not require attending intervention

15. Examples of Competency Rating Scales • Competent • The resident lacks smoothness and efficiency but adapts knowledge,judgement,and skill to start the case and induce anesthesia with no significant errors and most appropriately • The resident does not require attending interevention

16. Examples of Competency Rating Scales • Becoming competent • The resident lacks smoothness and efficiency and some knowledge,judgement or skill and exhibits some limits in adaptability. • The resident makes some signifiant errors but is able to start the case although not most appropriately • May require attending intervention

17. Examples of Competency Rating Scales • Not competent • The resident can only start the case and induce anesthesia with extensive coaching.The resident requires attending intervention

18. APA Model Logistics • Scheduled to occur at the 4,8,and 12 month intervals for the CAY1 and CAY3 residents • Scheduled to occur at the end of each subspecialty month for theCAY2 residents • Requires 3 different cases and faculty members

19. APA Model • Goal is for all residents to be competent or above • Becoming competent can be acceptable at start of new rotation but need to see improvement otherwise,remediate • Not competent=remediation/probation/ • Daily evaluations til competent or no credit

20. APA Model Pros • Helps to simply define competency • Residents know what is expected of them and what goals are • Helps faculty gauge competence in a uniform way • Easy form to fill out • Helps faculty give timely and constructive feedback • Works well with one-to-one system

21. APA Model Cons • Can be difficult to match patient /case complexity to resident year • Faculty/resident cooperation to assure timeliness of assessments • Residents free to pick and choose attendings who evaluate them • Residents(and faculty)wait until they have a perfect case to assess • Creates paperwork

22. Future of APA Model • Randomize assessments • Assessment teams • Expand concept to use for resident evaluation of faculty teaching