1 / 29

Portfolio Assessment in Clerkship

Portfolio Assessment in Clerkship. Michelle Gibson - Geriatrics (thanks to Chris Frank and Melissa Andrew too). Objective. Participants will discuss the use of portfolios in assessing medical trainees (after a brief review of some basic principles…). Background.

jill
Download Presentation

Portfolio Assessment in Clerkship

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Portfolio Assessment in Clerkship Michelle Gibson - Geriatrics (thanks to Chris Frank and Melissa Andrew too)

  2. Objective • Participants will discuss the use of portfolios in assessing medical trainees (after a brief review of some basic principles…)

  3. Background • Change to the Internal Medicine clerkship in January 2007 • 4 weeks to 6 weeks • Opportune time to review the assessment process in geriatrics • Previously, 8 assessment cards filled out by MD supervisors

  4. Challenges with old system • Cards were often filled out the last week of the rotation • Depended solely on supervisor impression • No direct observation was incorporated • Students were assessed using only one modality

  5. Reasons for change • Desire for more formative assessment opportunities • Desire for more robust assessment • (Need to meet course requirements!)

  6. Formative assessment • Assessment that occurs during instruction to provide feedback to teachers and students • Describes needs for future learning • Assessment for learning

  7. Summative Assessment • Assessment that occurs at the end of an instructional unit to document student learning • Certifies learning • Assessment of learning • Is this portfolio formative or summative?

  8. High Quality Assessment • Clear & appropriate learning targets • Appropriateness of assessment methods* • Validity • Reliability • Fairness • Positive consequences* • Alignment* • Practicality & efficiency*

  9. Reliability & Validity • Reliability: concerned with the consistency, stability, and dependability of the scores • Validity: the appropriateness of the inferences, uses, and consequences of the assessment • Is it assessing what you want it to assess?

  10. Portfolio • A systematic collection of student products to assess progress • Structured portfolio • The assessor determines the mandatory content of the portfolio

  11. Literature review • Assessment of housestaff in clinical rotations • Portfolio literature reviewed • Specific instruments examined • Portfolio components were selected to provide multiple methods of assessment for each clerkship objective

  12. Components by Methodology

  13. Mini-CEX

  14. TAB

  15. Card

  16. Medication Review - written • Based on an actual patient, preferably one for whom the student has responsibility • brief summary of patient, • meds he/she is taking including whether or not they are appropriate • suggestion of other medications he/she should be taking

  17. Self-assessment of write-up • H & P from early in rotation • Students asked to read and identify • 2 things done well • 2 things that could be improved • Reviewed with undergrad coordinator on the last day

  18. Components & CPE Categories

  19. Process • Orientation with students on Day 1 • Team, faculty, & residents are oriented to the portfolio • Students meet with undergrad coordinator at mid-term, review progress, identify any gaps, etc. • Last day - review portfolio with students, completion of CPE

  20. Evaluation of Pilot • Focus on acceptability • Students and assessors • Student evaluations ongoing - not seen by the undergrad coordinator until collated in April 2008 • Will redo faculty portion in December 2007

  21. Challenges noted - 1st time • First rotation ever for clerks • Time spent learning how to be a clerk • Couldn’t assemble half the portfolio in 3 weeks • Exam stress “detracted from our learning” • Supervisors were absent more than usual *all except exam stress have resolved

  22. Strengths noted • Multidisciplinary team members provided high-quality feedback • Direct observation occurred • Forms were felt to be user-friendly, acceptable • Students appreciated discussing their medication review and being observed

  23. Assessor ratings of forms • TAB • 5/5 for ease of use; 5/5 for usefulness • Mini-CEX • 5/5 for ease of use; 4/5 for usefulness • Time spent by assessor: 13.5 minutes • Encounter card • 4.5/5 for ease of use; 4.5/5 for usefulness • Time spent by assessor: 15 minutes

  24. My time? • 5-10 minutes on day 1 for orientation • Plan to develop “talking power point” • 15 min per student at the end of week 3 • Mid-term review • Could be any faculty • 15 min per student on the last day • Final evaluation • Could be any faculty

  25. My *impressions* • Students receive more feedback in a timely fashion • Direct observation doesn’t take long, and is very valuable • TAB form is very useful, and team members are pleased to be filling them out • Mid-term meetings are very useful

  26. Future directions • Continue with student and assessor feedback • Incorporate other tools? • Peer assessment? • Patient assessment? • Study it more • Implement it for residents

  27. Factors to note • Small unit, discrete group of nursing and other staff • Small division • Very workable for us … but for others?

  28. Questions to consider • Is this high quality assessment? How could it be improved? • Can it be extended into other areas? • Is there a role for a “whole clerkship” portfolio? “Pre-clerkship” portfolio? • Residents & non-MDs as assessors? • Electronic?

  29. Comments about our students • “ .. Is very attentive… he listens well to pt. & staff, & then goes off & evaluates what he’s been told” • “excellent verbal communication observed with all team members” • “better H & P than most residents”

More Related