Portfolios: Measuring the Art and the Science in Medical Education

1. Portfolios: Measuring the Art and the Science in Medical Education Carol Carraccio, MD, MA ABP R3P Project February 1, 2007

2. Objectives: • Become familiar with the contents of a learning portfolio • Integrate the components into a holistic approach to learner assessment

3. Components of the Portfolio

4. Global Assessments • Objectives for learning activities • Match items for assessment to the specific learning objectives for the given clinical experience/rotation • Example: Demonstrating the ability to manage a ventilator while in the PICU

5. Global Assessments • Benchmarks for Competencies • Performance indicators and standards, based on learner’s level of experience, that define competence in each of the ACGME domains • Example: Patient care requires gathering essential and accurate information • Benchmark: Perform a detailed and accurate PE with sensitivity to developmental, psychosocial and cultural issues • Standards: PGY1s expected to do for routine patients, PGY2s for most patients and PGY3s for almost all patients regardless of acuity/complexity

6. Components of the Portfolio

7. Task-oriented Assessment Tools • Structured tools • Complement global evaluations • Examples: • History & Physical Checklist or mini-CEX • Journal Club evaluation that addresses specific criteria and standards

8. Components of the Portfolio

9. Learning Plans • Individual learning plans • Require mentor guidance • Incorporate feedback from others and self-assessment • Creation of learning objectives and strategies to achieve them • Should ultimately become a living document that serves as the learner’s roadmap

10. Components of the Portfolio

11. Journal • Debrief from clinical experiences • Confidential vs assigned readers • Assigned by resident • Assigned by program director

12. Components of the Portfolio

13. Threaded Discussion • Ongoing bidirectional feedback between mentor and mentee • Recorded within the portfolio unlike email which is deleted

14. Components of the Portfolio

15. Critical Incidents • Instant evaluations documenting particularly positive or negative behaviors • Learner responds with a written “reflection” on how this incident will change future practice

16. Components of the Portfolio

17. Multi-source Feedback • Solicits feedback from all stakeholders • Delivered to residents in aggregate form by categories of evaluators • Mentors guide residents in incorporating this feedback into self-assessments and learning plans

18. Components of the Portfolio

19. Projects • Engage learners in tasks of real world practice • Address benchmarks for competencies that are not well demonstrated in the context of routine clinical care • Examples • Personal/professional quality improvement project • System errors activity such as root cause analysis

20. Components of the Portfolio

21. Logs • Conference attendance • ACGME logs • Continuity Clinic • Procedure

22. Building the Portfolio Integration of the Components

23. Intended Learning (Curriculum & Assessment) • Learning objectives for clinical skills • Benchmarks for competencies • Global &Task oriented assessments

24. Reflective Learning • Journal Writing • Threaded Discussions • Self-Assessment • Learning Plans

25. Putting It All Together …. My Reflective Learning My Intended Learning • Multi-source feedback • Critical • Incidents • Logs

26. My Professional Development • Maintenance of Certification • Continuous Personal & Professional • Development

27. Supporting the Formation of Professionals My Reflective Learning My Intended Learning My Professional Development