Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education

1. Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education Stacey L. Holman, MD, Sonya S. Erickson, MD, Diane M. Magrane, MD, Florencia G. Polite, MD, Joseph L. Hagan, ScD, Amy E. Young, MD Louisiana State University Health Sciences Center - New Orleans, Louisiana

2. Background: The ACGME has set forth new regulations regarding resident participation in meaningful quality improvement (QI) and patient safety initiatives. As training programs strive to meet these new requirements, it is evident that many residents have little prior education or exposure to the basics of QI. Objective: The purpose of this research project was to determine baseline resident knowledge of and attitudes towards quality improvement initiatives and patient safety. This information will be used to create and guide development of a resident curriculum.

3. Materials and Methods: • The survey instrument was created with two sections in mind -- • assessment of resident attitudes toward and involvement in quality improvement processes in training institutions • determination of baseline knowledge regarding QI • Quiz questions were developed using examples from question banks and teaching modules including: • CREOG patient safety modules • LSU Graduate Medical Education core compliance modules • Quality Improvement Steering Committee (EQuIP) teaching tools

4. Material and Methods: • The following QI domains of information were identified and included – • basic concepts • communication • health literacy & cultural awareness • medication safety • surgical safety • tracking & reminders • the impaired physician • disclosure of adverse events • The survey was also designed to examine relationships • between resident perception of quality improvement and • knowledge level.

5. Material and Methods: • The survey was administered electronically and anonymously to residents from OBGYN training programs across Louisiana (n=110) utilizing Survey Monkey™. • 80% was selected a priori as a “passing score” on the baseline knowledge quiz. (total score) • The Wilcoxon Signed Ranks test and Spearman’s Correlation Coefficient - compared scores for each domain of quality improvement to the mean total score. • The correlations of residents’ attitudes and house officer level • with the extent of QI knowledge were assessed - • Spearman's Correlation Coefficient and Analysis of Variance.

6. Results: • Response rate was 43.6% (N = 48). • Mean percentage of correct answers on knowledge quiz was 57%. • There was no statistically significant correlation between house officer • level and mean total score (p = 0.866).

7. Results: Domains of QI were each tested, and mean scores were all below the acceptable percentage (80% correct).

8. Results: The following chart shows the statistically significant domains as compared to mean total score of 57%. (p< 0.05)

9. Correlation Results: • Higher mean scores approached statistical significance when the QI initiatives were perceived as recently introduced vs. regularly scheduled into the curriculum (p = 0.062). • Communication scores were significantly higher when the QI initiatives were perceived as recently introduced QI vs. regularly scheduled into the curriculum (p=0.040). • Surgical Safety scores were negatively correlated with the following resident beliefs – • - “I can have a meaningful role in QI project implementation” (p=0.031). • “I have the ability to develop new QI projects that are meaningful • to my training” (p=0.024). • - No other domain was significant.

10. Conclusions: • The results of this needs assessment and knowledge test indicate that resident education is needed at all training levels. • The domains of impaired physician and tracking/reminder systems should be prioritized for curriculum development. • Future curriculum developments: • electronic design to continue with high response rate • tool distributed on a larger scale to gather multi-disciplinary data for our institution’s baseline knowledge as well as tracking improvement over time • integration of residents into meaningful QI/PS projects

11. Any efforts to teach quality improvement within training programs will likely increase knowledge and improve attitudes about quality improvement.

12. Acknowledgements: Drs. Erickson and Magrane – my APGO advisors for this project Local mentor, Dr. Amy Young, for her support of the APGO Scholars and Leaders Program Dr. Florencia Polite, Dr. Zee Ali, and Dr. Sheila Chauvin for their collaboration on this project Joe Hagan for his guidance with statistics and results

13. References: American Congress of Obstetricians and Gynecologists, Quality and Safety in Women’s Health Care, 2nd edition. Washington, D.C.: ACOG, 2010. CREOG Education Committee in conjunction with the American Congress of Obstetricians and Gynecologists, “Patient Safety Program for Residents”, released 2011. Enhancing Quality Improvement for Patients Steering Committee, Interim Louisiana Hospital, “Resident Training Modules in Quality Improvement”, released 2013. Joint Commission, National Patient Safety Goals 2013 & Sentinal Event Policy and Procedures. http://www.jointcommission.org Tomolo, AM, et al. “A case study of translating ACGME practice-based learning and improvement requirements into reality: systems quality improvement projects as the key component to a comprehensive curriculum.”Postgraduate Medical Journal 85 2009: 530-537. Tomolo, AM, et al. “Pilot Study Evaluating a Practice-Based Learning and Improvement Curriculum Focusing on Development of System-Level Quality Improvement Skills.”Journal of Graduate Medical Education 2011.

14. Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education Stacey L. Holman, MD, Sonya S. Erickson, MD, Diane M. Magrane, MD, Florencia G. Polite, MD, Joseph L. Hagan, ScD, Amy E. Young, MD Louisiana State University Health Sciences Center - New Orleans, Louisiana INTRODUCTION CONCLUSION RESULTS Results: Response rate was 43.6% (N = 48). Mean percentage of correct answers was 57%. There was no statistically significant correlation between house officer level and mean total score (p = 0.866). Domains as compared to Mean Total Score of 57% (p< 0.05) Domains of QI were tested, 8 total, and mean scores were all below the acceptable percentage (80% correct). • Results continued: • Correlation between resident attitudes and quiz scores included: • Higher mean scores approached statistical significance when the QI initiatives were perceived as recently introduced vs. regularly scheduled into the curriculum (p = 0.062). • Communication scores were significantly higher when the QI initiatives were perceived as recently introduced QI vs. regularly scheduled into the curriculum (p=0.040). • Surgical Safety scores were negatively correlated with the resident belief that “I can have a meaningful role in QI project implementation” (p=0.031). No other domain was significant. • Surgical Safety scores were negatively correlated with the resident belief that “I have the ability to develop new QI projects that are meaningful to my training” (p=0.024). No other domain was significant. • Conclusion: The results of this needs assessment and pilot knowledge test indicate that resident education is needed at all training levels. The domains of impaired physician and tracking/reminder systems should be prioritized for curriculum development. Any efforts to teach quality improvement within training programs are likely to increase knowledge and improve attitudes about QI. • Future curriculum development will focus on use of electronic design since there was a relatively high response to the electronic format. • The tool will be distributed on a larger scale to gather multi-disciplinary data for our institution’s baseline knowledge as well as tracking improvement over time. • References: • American Congress of Obstetricians and Gynecologists, Quality and Safety in Women’s Health Care, 2nd edition. Washington, D.C.: ACOG, 2010. • CREOG Education Committee in conjunction with the American Congress of Obstetricians and Gynecologists, “Patient Safety Program for Residents”, released 2011. • Enhancing Quality Improvement for Patients Steering Committee, Interim Louisiana Hospital, “Resident Training Modules in Quality Improvement”, released 2013. • Joint Commission, National Patient Safety Goals 2013 & Sentinal Event Policy and Procedures. • http://www.jointcommission.org • Tomolo, AM, et al. “A case study of translating ACGME practice-based learning and improvement requirements into reality: systems quality improvement projects as the key component to a comprehensive curriculum.”Postgraduate Medical Journal 85 2009: 530-537. • Tomolo, AM, et al. “Pilot Study Evaluating a Practice-Based Learning and Improvement Curriculum Focusing on Development of System-Level Quality Improvement Skills.”Journal of Graduate Medical Education 2011. • Acknowledgements: • Thanks to Drs. Erickson, Magrane, and Young for advising this project. • Thanks to Drs. Polite and Ali, Sheila Chauvin, and Joe Hagan for collaboration and guidance on this project. • Background: The ACGME has set forth new regulations regarding resident participation in meaningful quality improvement (QI) and patient safety initiatives. As training programs strive to meet these new requirements, it is evident that many residents have little prior education or exposure to the basics of QI. • Objective: The purpose of this research project was to determine baseline resident knowledge of and attitudes towards quality improvement initiatives and patient safety. This information will be used to create and guide development of a resident curriculum. • Materials and Methods: • Survey was created with two sections: • assessment of resident attitudes and involvement in quality improvement processes in training institutions • determination of baseline knowledge regarding QI • Survey was designed to examine relationships between resident perception of quality improvement and knowledge level. • Quiz questions were developed from existing question banks and teaching modules including: • CREOG patient safety modules • LSU Graduate Medical Education core compliance modules • Quality Improvement Steering Committee teaching tools (LSU Interim Hospital) • QI domains included: basic concepts, communication, health literacy & cultural awareness, medication safety, surgical safety, tracking & reminders, the impaired physician, and disclosure of adverse events. • Survey was administered electronically and anonymously to residents from OBGYN training programs across Louisiana (n=110) utilizing Survey Monkey™. • 80% was selected a priori as a “passing score” on the baseline knowledge quiz. • The Wilcoxon Signed Ranks test and Spearman’s Correlation Coefficient were used to compare scores for domains of quality improvement to the mean score. • The associations of residents’ attitudes and house officer level with the extent of QI knowledge were assessed using Spearman's Correlation Coefficient and Analysis of Variance. Department of Obstetrics and Gynecology; Louisiana State University Health Sciences Center – New Orleans Stacey L. Holman, MD sholma@lsuhsc.edu APGO/ Scholars Program March 2013