390 likes | 519 Views
Residents In Trouble. M. Rebecca Hoffman, MD, MSPH December 8, 2008 Academy for Scholarship in Education. Overview. Scope of the Problem Literature Review Study Design Study Results (in progress). Scope of the Problem. The goal of Family Medicine Residency Residents’ backgrounds
E N D
Residents In Trouble M. Rebecca Hoffman, MD, MSPH December 8, 2008 Academy for Scholarship in Education
Overview • Scope of the Problem • Literature Review • Study Design • Study Results (in progress)
Scope of the Problem • The goal of Family Medicine Residency • Residents’ backgrounds • No guidance
What does the RRC say? • Must have advising system • Must give regular feedback BUT… no real guidance regarding what/how
Consequences of RIT • For the individual RIT • Psychological stress • Financial stress • Eventual job security • For the program • Adequate coverage of duties • Morale • Faculty time/energy • Dismissal
Consequences… • Other consequences • Recruiting • Impact on patient care
Literature Review • Not much on the prevalence of RIT • Reamy et al (1): 25 year in-depth study of one family medicine residency program • 9.1% • Yao and Wright (2): Survey of Internal Medicine PDs– • 6.9% • Williams et al (3): SIU Gen Surgery Residency • 22%
Kinds of Problems • Knowledge Deficits • Attitudinal Problems • Interpersonal Conflict • Psychiatric Illness • Substance Abuse • Family Stress • Relationship Disruption
Other Classification • Academic Performance • Classroom, test • Clinical Performance • Applied knowledge/skills • Professional Behavior • Workplace behavior, relationships with other professionals, etc • Others • Difficult to classify
So we can name it… • But what do we do with the information? • Literature reviews: • “Need to do better” • Core content review, quizzes • Counseling • Treatment of illnesses • Repetition (rotation, year) • Probation • Dismissal • Others
Summary of the problem • We know it’s there, but we aren’t sure exactly how common it is • We aren’t good at predicting who will struggle • Once we identify them, we don’t know what to do with them
RIT Family Medicine Study Primary Aims 1) Determine the prevalence of RIT in the SIU Family Medicine Residency Programs 2) Describe and categorize the types of troubles encountered in the FCM residency programs
RIT Aims... 3) Identify and describe the kinds of processes the programs employ to address deficiencies with RIT 4) Describe outcomes of the residents who underwent remediation 5) Identify early indicators to predict RIT
RIT Study Design • Chart review/resident record -based • Prevalence over significant timeframe • 10 years of entering classes • Entering class of 1993-entering of 2003 • Why? • De-identified data from all 4 programs • To further assure confidentiality • No one program singled out
RIT Study • Initially modeled after Reamy et al study (1) after discussion with Dr. Reamy about the process in his residency program • Assistance from Reed Williams, PhD and Nicole Roberts, PhD
Methods • Rosters of all entering residents for the 10 years from all programs • Random ID numbers assigned • Records from each program reviewed in their entirety • Mostly paper • Some electronic
Methods… • Data collection instrument created based on personal experience, literature, and discussion with Reed Williams and Nicole Roberts • Feedback from academic faculty • Suggestions incorporated as needed
Collected for all residents: • US Medical School Grad (Y/N) • USMLE or NBOME or COMLEX scores • Step 1, Step 2, Step 3 (when available) • Gender (M/F) • ITE Scores • PGY1, PGY2, PGY3 (when available) • Did the resident have substantial difficulties? (Y/N)
For those who struggled • Time frame- when first identified • Categorization of the main problem • Academic, Clinical, Professionalism, Other • Main performance area involved • See list • Breadth of performance areas • Academic, Clinical, Professional, Other • All performance areas involved • See list
For those who struggled • What remediation attempted? • See list • What was the final resolution? • Graduated with concerns • Graduated without concerns • Voluntarily left • Dismissed • Finished on probation • Not documented
Severity Rating • Compared to other residents with performance problems, this resident’s problems were: • Likert scale, 1-5 • 1 among the least serious • 5 among the most serious • Narrative descriptions of overall case, time course, etc
Example of Survey Instrument • Survey Monkey for data entry
Data Collection • 2 raters • Independently reviewed same 5 charts at first program to establish inter-rater reliability, then 3 charts at each other program
Data Collection… • Then independent record review • Any case the reviewer identified as a RIT was then reviewed by the other reviewer independently, ratings discussed after each case • “Borderline cases” also reviewed by both
Data Analysis • Data entered into Survey Monkey for ease of collection • Analysis: descriptive and basic statistics • When all data complete, analysis using SPSS
Results • 96 Residents reviewed (as of 12/4/2008) • US Med School Grads: 69% (66) • Male: 65% (62) • Female: 35% (34) • Prevalence of documented RIT: 38.5% (37)
RIT (N=37) • US graduates: 59.5% (22) • Foreign graduates: 40.5% (15) • Male: 70.3% (26) • Female: 29.7% (11) • Male, US: 40.5% (15) • Male, Non-US: 29.7% (11) • Female, US: 18.9% (7) • Female, Non-US: 10.8% (4)
Most Important Single Problem Area? • Knowledge (10) • Putting everything together (3) • Data interpretation/diagnosis (2) • Data collection (2) • Incomplete paperwork/charts (2) • Lack of motivation/interest (2) • Treatment/management (2)
Breadth of problems • Number of performance areas involved ranged from 1 to 23 • Average: 7 • See breakdown of individual RIT on handout
Remediation • Ranged from “none” or “told to improve” to 12 or more interventions for a single resident • Most commonly reported: Increased meetings with advisor/mentor
Outcomes of RIT (residency) • 26 graduated from the program (70.3%) • 11 did not graduate (29.7%) • Left voluntarily: 6 • Changed specialty: 2 • Dismissed: 3 • More complete follow up not complete • Have initiated licensure status and board certification F/U but too little data to report
Conclusions? • RIT more common than suspected based on lit. • Increased prevalence compared to other studies • ? Over-diagnosis • For “mild cases” • ? Truly higher prevalence • Due to population? • Due to program characteristics? • All RIT who had concerns in more than 8 areas had ongoing concerns at graduation or did not complete program • RIT who struggled in all 3 performance areas also either did not complete or graduated with ongoing concerns • Professionalism issues very common
Problems Encountered • Information in files varies between programs • Missing data • Esp. test scores (fortunately, may be able to acquire) • Even within programs, varying data • Classification of single most important area for major problem residents • We collected data for an additional group of residents: those with ITE performance less than 10th %ile, even if no other problems, will follow those as well (n=4 right now)
To come! • Full data reporting (approx 270) • Including USMLE and ITE data • Predictive model? • Follow up data (post-graduation)
Special Thanks to: • Nicole Roberts, PhD data spelunker extraordinaire • Reed Williams, PhD • Steve Verhulst, PhD • Jerry Kruse, MD, MSPH • SIU Family Medicine Residency Program Directors: • Janet Albers, MD • John Bradley, MD • Tom Miller, MD • Penny Tippy, MD • Wiley Jenkins, PhD
Referenced Articles • 1. Reamy BV, Harman JH. Residents in trouble: an in-depth assessment of the 25-year experience of a single family medicine residency. Fam Med 2006;38(4):252-7. • 2. Yao DC, Wright SM. National survey of internal medicine residency program directors regarding problem residents. JAMA 2000;284:1099-1104. • 3. Williams RG et al. The nature of general surgery resident performance problems (publication pending)