1 / 39

Assessing Professionalism

Assessing Professionalism. Ramona Kearney MD, MMEd , FRCPC University of Alberta. How to reference this document: Kearney R., Resuscitating Professionalism in Your Program. CanMEDS Train-the-Trainer Program on Professionalism. 2009. Objectives.

bendek
Download Presentation

Assessing Professionalism

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Professionalism Ramona Kearney MD, MMEd, FRCPC University of Alberta How to reference this document: Kearney R., Resuscitating Professionalism in Your Program. CanMEDS Train-the-Trainer Program on Professionalism. 2009

  2. Objectives At the conclusion of this session you will be able to: • Define CanMEDS Professional competencies • Describe 3 practical methods for assessing professionalism • Integrate one new method of assessment into your program

  3. Definition: Professional Role • As Professionals, physicians are committed to the health and well-being of individuals and society through • Ethical practice • Profession-led regulation • High personal standards of behaviour

  4. Physicians have a unique role in society • We are: • dedicated to the health and caring of others, • through mastery of a body of knowledge, skills and attitudes and • guided by a code of ethics and a commitment to clinical competence

  5. Key competencies of the Professional Role • Demonstrate a commitment to their patients, profession and society through ethical practice • Demonstrate a commitment to their patents, profession and society through participation in profession-led regulation • Demonstrate a commitment to physician health and sustainable practice

  6. Enabling Competencies Demonstrate to their patients, profession and society through ethical practice • Exhibit appropriate behaviours – humanistic qualities • Maintain competence • Practice in an ethical and legally responsible manner

  7. Enabling Competencies Demonstrate a commitment to their patients, profession and society through participation in profession-led regulation • Be accountable to professional bodies • Fulfill legal obligations • Be aware of codes of practice and follow them • Do not tolerate colleagues’ unprofessional behaviors • Participate in peer review

  8. Enabling Competencies Demonstrate a commitment to physician health and sustainable practice • Balance your personal/professional priorities • Strive for ongoing personal development • Assist colleagues in need

  9. Task • List the tools that can be used to assess professionalism. • How suitable is each tool for this task?

  10. Key Tools for Assessing theCanMEDS Competencies ITER= In-Training Evaluation Report OSCE = Objective Structured Clinical ExaminationSP = Standardized patient 360° = Multi-source feedback

  11. Knowledge tests • A cognitive base for professionalism exists • Testing this knowledge underpins its importance • SAQ format likely most useful

  12. OSCE and Simulations • can be used through a station specifically addressing professionalism • can be useful for assessing teams and inter-professional activities • performance can be confounded by the resident’s communication skills • measures what the resident can do under testing circumstances rather than what the resident actually does in practice • is a single measure reflecting the specific conflict in a testing context at the time of the measurement

  13. Direct Observation and In-Training Evaluation Reports A form documenting a supervisor’s evaluation of the resident based on a defined clinical experience over a defined timeframe. • consists of a set of Likert scales, categorical lists, checklists, or global assessment scales based on observed clinical performance • includes broad evaluation of content and behaviour areas • efficient • evaluator can use multiple sources of information • providessummative evaluation data

  14. Direct Observation and In-Training Evaluation Reports Disadvantages: • difficult to assess specific skills/ knowledge content areas • prone to halo effect (overall impression influences specific scores in each area) • demonstrate leniency bias (evaluators not wanting to provide negative evaluation) • range-restriction bias (evaluators using small portion of the scale for most evaluations)

  15. Direct Observation (ITERs) Key Points: • Training evaluators is important to establish reliable expectations and interpretations of the forms • Forms and rating scales should be specific to the expected accomplishments during the clinical period in question • Correlation between clinical excellence and good professional behaviours (Rhoton, Papadakis)

  16. Task What are the professional behaviors you want to assess?

  17. Attributes • Integrity (morally upright, honourable, conscientious, diligent) • Respectfor patients’ views, dignity and privacy • Respectfor colleagues and co-workers • Altruism (unselfish regard or concern for others as a principle of action) • Accountabilityfor personal actions (towards patients, society and the profession) • Responsibility (reliable, punctual, available, follows through on tasks)

  18. Attributes • Empathy (understanding of and concern for the patient’s condition, compassion • Trustworthy (inspires confidence in the goodness of reliability of oneself, honest) • Maintains confidentiality • Adheresto ethical and legal codes • Unbiased (free of inappropriate prejudice) • Self-awareness (demonstrates self-assessment through insight into one’s character and capabilities)

  19. Attributes • Maintains balancebetween personal and professional roles (includes prioritizing tasks, time, events; and dealing with stress) • Copeswith uncertainty and error (appreciates the possibility of uncertainty in decision-making and has strategies for dealing with error) • Acceptscriticism appropriately • Commitment to lifelong learning • Mentor (ability and willingness to teach and guide, awareness of being a role model)

  20. Attributes and Behaviour Attributes can be meaningfully assessed in terms of observable behaviours which are seen in the workplace Examples of acceptable and unacceptable behaviours help assessors identify residents who are potentially unprofessional

  21. Professionalism Mini-Evaluation Exercise • Doctor-patient relationship skills • Reflective skills • Time management • Inter-professional relationship skills

  22. Physician Health and Sustainable Practice • A new component of professionalism • A complex and multifaceted issue • Related to excellence in patient care, access, and system efficiency • Requires formative (and sometimes) summative feedback • Formal educational methods required • Challenges of evaluation

  23. Behaviours (specific to identifying the unhealthy practitioner) • Insightful practice vs. insecurity/defensiveness • Healthy vs. unhealthy coping strategies • Emergence of new or difficult behaviour (irritability, anger, neutrality, flippancy, sadness, cynicism) • Disruptive vs. collaborative style • Reasonable restfulness vs. exhaustion • Engagement vs. isolation • Appropriate management of physician-patients vs. discomfort, crossing boundaries or violations D. Puddester, 2006

  24. 360° assessment (Multi-source Feedback) Assessments completed by many people using checklists or rating scales to assess various competencies; evaluations are summarized to provide anonymous feedback. • Based on actual behaviour in a clinical setting • Includes multiple sources of information • Provides information on overall practice patterns • Good reliability if done properly • Communication and interpersonal skills • Professionalism

  25. 360°/ Multi-source Feedback Disadvantages: • Reliability dependent on high number of assessors • Difficult to ensure assessors understand the process • Large human resource requirement • Needs constant monitoring for effectiveness and appropriateness of feedback data

  26. 360°/ Multi-source Feedback Key Points • Need at least 5-10 evaluations (people) from each perspective to ensure reliability • Assessment instruments must be simple • Good for formative feedback rather than summative evaluation or pass/fail decisions • Learners should not be able to identify individual sources of the evaluations

  27. Task • Which groups (people) will be the key evaluators in your specialty? • For which professional behaviours will each group provide useful information?

  28. Groups • Physicians (within and outside your own faculty) • Non physician colleagues/coworkers • Nurses • Patients • Students • Peers

  29. Portfolios Learners record their experiences, their self-assessment of the experiences and define and update learning goals. • Includes a variety of supporting documentation reviewed periodically by supervisor/program director • Can sample a large breadth of content areas • Promotes self-assessment and goal-setting • Can identify areas of deficiency in individual experience and program design • Can assess progress over time • Based on experiential learning principles

  30. Portfolios Disadvantages: • Time consuming for data entry • Difficult to standardize and assess accuracy and completeness • Time consuming for program directors to review • Not good for pass/fail decisions or summative evaluations

  31. Portfolios Key points: • Unknown how accurate resident records are • Relies on learner initiative

  32. Task • Which experiences of the learner should be considered in the portfolio to demonstrate learning of Professionalism? • What material (evidence) should be collected?

  33. Portfolio Material • Critical incidents of events with patients • Routine clinical experiences • Video recordings of consultations • Critical reviews of articles • Literature and film reviews • Feedback material • A reflective journal • Examination results

  34. University of Alberta Anesthesiology Residency Program Professional Activity Portfolio RESIDENT’S NAME: Residents - It is your responsibility to maintain an accurate Portfolio. Under “details & reflections” please include dates, topics, self-assessment, feedback received, your own reflections, etc. A copy will be inserted in your academic file & reviewed at the semi-annual interview with your program director.

  35. How to assess a portfolio • Criterion referencing • Explicit benchmarks for measurement • Relate measurements to learning outcomes • Determine the validity of the evidence • Ensure learning is authentic/personalized

  36. Examples of tools • CPSA website (www.cpsa.ab.ca) • – PAR program www.par-program.org • ACGME outcome project • www.acgme.org/outcome/implement • www.acgme.org/outcome/assess • RCPSC publications – Special projects and reports – Professional role document • www.rcpsc.medical.org/ publications • Portfolios • Simon Cotterill, University of Newcastle, Managed Environments for Portfolio-based Reflective Learning www.eportfolios.ac.uk/eportfolio • Ramona Kearney, University of Alberta; Portfolios for Anesthesiology residents • P-Mini Evaluation Exercise (P-MEX) – Cruess et al ( in package)

  37. Reflection • Take a few minutes to think about which assessment methods are most appropriate for your institution and write list them.

  38. Action Plan • Take a few minutes to reflect on this half-day, and complete the relevant section of the action plan

More Related