Portfolios: Measuring the Art and the Science in Medical Education

Portfolios: Measuring the Art and the Science in Medical Education Carol Carraccio, MD, MA ABP R3P Project February 1, 2007

Objectives: • Become familiar with the contents of a learning portfolio • Integrate the components into a holistic approach to learner assessment

Components of the Portfolio

Global Assessments • Objectives for learning activities • Match items for assessment to the specific learning objectives for the given clinical experience/rotation • Example: Demonstrating the ability to manage a ventilator while in the PICU

Global Assessments • Benchmarks for Competencies • Performance indicators and standards, based on learner’s level of experience, that define competence in each of the ACGME domains • Example: Patient care requires gathering essential and accurate information • Benchmark: Perform a detailed and accurate PE with sensitivity to developmental, psychosocial and cultural issues • Standards: PGY1s expected to do for routine patients, PGY2s for most patients and PGY3s for almost all patients regardless of acuity/complexity

Task-oriented Assessment Tools • Structured tools • Complement global evaluations • Examples: • History & Physical Checklist or mini-CEX • Journal Club evaluation that addresses specific criteria and standards

Learning Plans • Individual learning plans • Require mentor guidance • Incorporate feedback from others and self-assessment • Creation of learning objectives and strategies to achieve them • Should ultimately become a living document that serves as the learner’s roadmap

Journal • Debrief from clinical experiences • Confidential vs assigned readers • Assigned by resident • Assigned by program director

Threaded Discussion • Ongoing bidirectional feedback between mentor and mentee • Recorded within the portfolio unlike email which is deleted

Critical Incidents • Instant evaluations documenting particularly positive or negative behaviors • Learner responds with a written “reflection” on how this incident will change future practice

Multi-source Feedback • Solicits feedback from all stakeholders • Delivered to residents in aggregate form by categories of evaluators • Mentors guide residents in incorporating this feedback into self-assessments and learning plans

Projects • Engage learners in tasks of real world practice • Address benchmarks for competencies that are not well demonstrated in the context of routine clinical care • Examples • Personal/professional quality improvement project • System errors activity such as root cause analysis

Logs • Conference attendance • ACGME logs • Continuity Clinic • Procedure

Building the Portfolio Integration of the Components

Intended Learning (Curriculum & Assessment) • Learning objectives for clinical skills • Benchmarks for competencies • Global &Task oriented assessments

Reflective Learning • Journal Writing • Threaded Discussions • Self-Assessment • Learning Plans

Putting It All Together …. My Reflective Learning My Intended Learning • Multi-source feedback • Critical • Incidents • Logs

My Professional Development • Maintenance of Certification • Continuous Personal & Professional • Development

Supporting the Formation of Professionals My Reflective Learning My Intended Learning My Professional Development

Portfolios: Measuring the Art and the Science in Medical Education