Methods:

1. On-Line Portfolio Implementation of 360 Degree Surveys to Assess Competencies in the Dartmouth-Hitchcock Leadership Preventive Medicine Residency (DHLPMR) Martha Regan-Smith MD, EdD; Tina Foster MD, MPH, MS; Mariah Capurso; Paul Batalden, MD Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire Introduction: DHLPMR, a residency which combines training in Preventive Medicine and other accredited GME training programs at DHMC, uses 360 surveys in an online portfolio as one method of resident assessment. • Results: • Prior to entering DHLPMR, low self-assessments in PBLI and SBP indicated that residents did not feel competent in these areas even after 2-5 years of clinical training. • Residents rated themselves more highly in all six competencies as they progressed through the program, with SBP showing the most improvement, followed by MK, ICS and PBLI. This is consistent with the program’s emphasis on understanding and improving care for small populations of patients. • Methods: • 360s are distributed before entering DHLPMR as a needs assessment and at the end of years 1 and 2. • Self-rating (1 never, 2 occasionally, 3 sometimes, 4 usually and 5 always) of each item by the resident is compared to the mean rating by each rater group. • Items showing differences between the resident self-assessment rating and rater group ratings are considered for the resident’s professional learning plan. • Lessons Learned: • The ease of use of an online evaluation system greatly affects response rates. • An invitation from resident to rater which includes a brief explanation of 360 methodology will increase response rates and quality. • Online e-mail messages requesting survey completion were sometimes automatically deleted as SPAM. • Too few raters compromised rater anonymity. • Some residents did not accept that differences between self-ratings and those by others were meaningful. • Timeliness in survey completion and ability to provide constructive feedback in answer to open-ended questions increased significantly after program leadership completed their own 360s. • Those who participated in their own 360 survey and received written feedback found the experience to be more useful for their professional development than feedback they received by other methods. • 360 survey items enable the residency leadership to define the values and priorities of the residency and describe a desirable learning environment.