1 / 176

Site Visit 102: The DIO’s Role for Program Site Visits

Site Visit 102: The DIO’s Role for Program Site Visits. Ann M. Dohn, M.A. DIO Stanford. Disclaimer. No Conflicts of Interest to Report . DIO Competencies. Maintaining the Institution’s Residency Program’s ACGME accreditation Improving the Institution’s education program

tudor
Download Presentation

Site Visit 102: The DIO’s Role for Program Site Visits

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Site Visit 102: The DIO’s Role for Program Site Visits Ann M. Dohn, M.A. DIO Stanford

  2. Disclaimer • No Conflicts of Interest to Report

  3. DIO Competencies • Maintaining the Institution’s Residency Program’s ACGME accreditation • Improving the Institution’s education program • Developing and supporting residency program directors

  4. DIO Vision • “Your job is not a powerful one…your job is to empower others.” i.e. program directors • Helen Rantz, 1984

  5. Session Objectives • To understand the role of the DIO in ACGME program site visits • Using ACGME annual resident and faculty surveys • Using internal reviews and annual program evaluation in preparation for site visits • Using other potential measures of program quality

  6. Session Objectives • A time of change… • Living in two worlds-starting 7/1/2013 • “PIF-less Site visits / NAS • Site visits with PIFs ACGME OLD NEW

  7. Then and Now • 1984 site visits vs. 2012 site visits

  8. Start with the New and “unknown” • Unknown is scary

  9. New Model: What Do We Know? • Longer Cycles • Annual WebADS Updates • ACGME Resident and Faculty Surveys • End of the PIF

  10. End of the PIF • DIOs will have more recreational reading time!

  11. “New” Model • Effective 7/1/2013 • “Early adaptors”= RRCs deciding to go “PIF-less” • Emergency Medicine • Internal Medicine • Neurosurgery • Orthopedic Surgery • Pediatrics • Radiology (Diagnostic) • Urology • and their subs

  12. New Accreditation System (NAS) • The NAS requires categorization of the ACGME’s Common Program Requirements to clearly identify: • Core Requirements • Detail Requirements • Outcome Requirements

  13. New Accreditation System (NAS) • Core Requirements • Statements that define structure, resource, or process elements essential to every graduate medical educational program. • Duty Hours • Sponsoring Institution’s responsibility for Program Accreditation • Change in program director • Qualifications of the program director

  14. New Accreditation System (NAS) • Detail Requirements: • Statements that describe a specific structure, resource, or process for achieving compliance with a Core Requirement.

  15. Examples of “Detail” Requirements • Detail Requirements include particular educational approaches and learning experiences • a given number of lectures or hours of bedside teaching experiences • Ensuring compliance with grievance and due process procedures as set forth in the Institutional Requirements and implemented by the sponsoring institution

  16. New Accreditation System (NAS) • Outcome Requirements: • Statements that specify expected measurable or observable attributes (knowledge, abilities, skills, or attitudes) of residents or fellows at key stages of their graduate medical education. • Programs in substantial compliance with the Outcome Requirements may utilize alternative or innovative approaches to meet Core Requirements.

  17. Example of “Outcome” Requirements • Outcome requirements include: • Milestones • Residents must be able to competently perform all medical, diagnostic, and surgical procedures considered essential for the area of practice.

  18. Common Program Requirements

  19. Neurosurgery – new Accreditation Example…

  20. New Model-Tracer Model-Inverted Model • Increases value on resident input • ACGME survey • Resident interviews • Focuses on PROGRAM rather than a description of the program • The Tolstoy novel will no longer be written!!

  21. New Model-PIF Questions • Small number of questions from PIF now in annual WebADS update • Remaining information assessed by the field staff

  22. Tracer Method • Allows for the selection of particular elements to be evaluated: • Citations • Complaints from residents/faculty • Non-compliance reported on the ACGME resident survey • Duty hour and learning environment standards

  23. Tracer Method • Focuses on: • Program’s actual operations • Implementation of policies and procedures

  24. Tracer Method – Interactive – A Key Change • Allows for discussion of important topics related to the new duty hour standards • Provides opportunities for site visitors to educate program leaders and share innovative practices

  25. Tracer Method • Longitudinal information on: • New standards • Prior citations/corrective actions • Resident survey identified areas of non-compliance • Areas identified during preparation for the Site Visit

  26. Tracer Method • Will focus on relevant high-priority issues for the given program • Planned for two-four areas/topics per program • Done during the regular interview (no walkarounds)

  27. Tracer Method • Enhancing resident input • Residents to directly respond to the site visitor with a single, confidential “consensus list” • strengths • areas for improvements • starts with the trainees

  28. PIF-Less Site Visit: The Stanford Experience • August 2011 • Three Programs • Visit done in lieu of mid-cycle internal review • Data not used for accreditation • Focus was on program requirements • Used comprehensive data system (no paper)

  29. PIF-Less Site Visit: The Stanford Experience (2) • Individuals interviewed = regular site visit • ACGME provided comprehensive reports • Extensive debriefing with the site visitors and program participants • Very positive experience for PDs. PCs, and the DIO and GME staff

  30. PIF-Less Site Visit: Stanford Findings • PIF-Less (inverted) site visit report finding • Comparable to standard internal review • Note: our protocol always starts with the residents/fellows

  31. PIF-Less Site Visit: ACGME Findings • Findings: • Residents feel more engaged • PDs love it • Early ID of Program Themes • Improves continuity and integration of data • Seamless process for follow-up • Increased opportunity for reconciliation of discrepant issues • Transformative Model - rather than additive

  32. PIF-Less Site Visit: ACGME Findings (2) Findings Continued • Comparable to visit with PIF for citation review • Superior for review of program improvements and positive program elements • Variability with other elements • Dependent on data system used by the institution/program

  33. Upcoming Changes in the Site Visit • Broader implementation of the inverted Site Visit • Team site visits (two site visitors) • Programs • Institutions

  34. Yikes! ACGME is coming….

  35. ACGME Use of Survey Data during the Site Visit • Use resident data to validate/clarify program information • Compare resident data to program director and faculty interviews • Validate resident data to program documentation • Compare duty hours responses to institutional data

  36. DIO Use of Data for Program Site Visits • ACGME resident and faculty surveys • APRs • Internal Reviews • Other measures of program quality

  37. Annual ACGME Surveys2012

  38. Annual ACGME Surveys2009

  39. ACGME Concerns with the Resident Survey • ACGME concerns with the resident survey: • “Honesty” in reporting • ? Residents being coached ? • Amount of time spent by residents completing the survey = 20 seconds per question • Survey questions being sold to companies who want to ‘help’ programs prepare • Survey fatigue

  40. ACGME Annual Resident Survey • Possible solutions to using the survey • 2012- Aggregation of individual survey questions into groupings • 2012-use of alternate forms of the survey • Educating residents about the vital nature of the survey

  41. Annual Program Reviews: Summary

  42. Annual Program Reviews: Data Elements

  43. Internal Reviews

  44. Monitoring Internal Reviews

  45. Other Possible Sources of Data • GME Annual House Staff Survey • Duty Hours Reports • Evaluations of programs, faculty, and house staff • Alumni surveys • Board and in-training exam scores

  46. GME Annual House Staff Survey • Provides feedback on eight (8) key areas of Graduate Medical Education: • Overall Experience • Program Curriculum • Program Faculty • Evaluation and Feedback • Training Environment • Personal Wellness • Quality Improvements • Communication and Patient Perceptions • Other areas….(anything we missed)

  47. GME Annual House Staff Survey • Provides a level of detail not available from the ACGME survey • Such as on call room issues, fear/intimidation issues, harassment issues, faculty • Allows for text answers, comments Warning: Don’t do a survey unless you are willing to accept and act on the input

  48. GME Annual House Staff Survey

  49. GME Annual House Staff Survey

  50. Duty Hour Reports

More Related