The reflective statement: A new tool to assess resident learning.

1. The reflective statement: A new tool to assess resident learning. Sean F. Monaghan, MD, Andrew M. Blakely, MD, Pamela J. Richardson, C-TAGME, Thomas J. Miner, MD, William G. Cioffi, MD, David T. Harrington, MD Association for Program Directors in Surgery Meeting March 26, 2011

2. Background - Portfolios • Enhances medical education by making the student an active learner • Requires an assessment of knowledge deficits, optimal method of learning, and creation of a plan for remediation. • Effective tool in resident and medical student education Challis, M. Med. Teach. 2001; Davis, M. H. Med. Educ. 2009; O'Sullivan, P. S., Adv. Health Sci. Educ. Theory. Pract. 2004.

3. Background – ACGME Core Competencies • Grouped reflective statements can can assess the ACGME core competencies • Communication • Knowledge • Patient Care • Practice Based Learning • Professionalism • Systems Based Practice • Difficult to assess these competencies Wasnick, J. D., Acad. Med. 2010; Yaszay, B., Orthopedics, 2009; Lurie, S. J., Acad. Med. 2009

4. Background – Educational Budget • Since work hour requirement were introduced, emphasis on the educational value of each hour • Future Demands • Electronic health record • Simulation • Further duty hour regulation • The collection of reflective statements can serve as a tool to assess the curriculum

5. Hypothesis The collection of reflective statements can serve as a tool to assess the curriculum

6. Methods • During two academic years the reflective statement was filled out prior to mandatory conference • Done approximately four times each year • Data de-identified as to who filled out the statement • Data categorized after all statements were collected

7. Methods • A prior categorized into: • ACGME core competency • Who taught the item • Faculty • Resident • Other • Where item was learned • Self • Clinic • Conference • OR • Wards • Comparison across categories above as well as PGY • Chi square analysis with bonferroni correction as needed • IRB approved

8. Results • 304 surveys of 309 were completed • 98.4% response rate • 43% of responses were completed by interns • Reflects their percentage in the program

9. Results – ACGME Core Competencies * *p<0.00625 compared to Knowledge

10. Results – ACGME Core Competencies # # # #p<0.00625 compared to Patient Care

11. Results – Competency learned by PGY

12. Results – Who taught the item * * *p<0.025 compared to Faculty

13. Results – Competency learned by PGY

14. Results – Learning from residents *p<0.001

15. Results – Where item was learned * * * *p<0.0125 compared to self

16. Results – Where learned by PGY

17. Discussion – Core Competencies • Patient Care and Knowledge most cited competencies • Why “soft” competencies (Com, Pro, SBL) not mentioned • Lack of knowledge • Not felt to be important • Did not realize they learned the item • Plan to correct • Portfolios to be reviewed with residents

18. Discussion – Who Taught • Faculty teaching • Across all PGY, most items taught by faculty • Faculty mentoring • Dependence on faculty for teaching highlighted in this assessment. • Need programs to enhance the teaching of faculty • Chief residents learn from other residents • Sharing of cases with peers

19. Discussion – Where Learned • Experiential learning • Wards • OR • Why such low self-learning • Not sure how much was done • May not be effective • May need to develop self learning skills • Lack skills lab learning • Much more learning done with patient in operating room

20. Study Deficits • Baseline analysis • Reflective statement not designed to assess entire program • Question may be misinterpreted

21. Conclusions • Knowledge and Patient Care are major core competency mentioned and faculty taught the most • Resident teaching had “U” shaped distribution • Self learning was a minor component • Reflective statement can be used to assess a program

22. Future Directions • Add number of hours spent self learning in the past week • Program director review of the statement with the residents

23. The reflective statement: A new tool to assess resident learning. Sean F. Monaghan, MD, Andrew M. Blakely, MD, Pamela J. Richardson, C-TAGME, Thomas J. Miner, MD, William G. Cioffi, MD, David T. Harrington, MD Association for Program Directors in Surgery Meeting March 26, 2011