Continuous Quality Improvement (CQI): Part 2

1. Continuous Quality Improvement (CQI): Part 2 Bethany Geldmaker PNP, PhD Tracy Jebo MPH

2. Learning Objectives Review with Infant Mental Health Advisory Board: Part 1-- Foundational Continuous Quality Improvement Results of the Self-Assessment Survey results Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback

3. Review Performance Management CQI PDSA Cycle

4. Performance Management and CQI Two primary components: • Performance Management: Using data to improve performance • CQI (Continuous Quality Improvement): Involving all stakeholders to examine the performance of a process and address identified gaps

5. Quality Improvement Uses data on how you are performing to drive activities which lead to true improvement. Utilizes stakeholders to address identified gaps continually improve processes Addresses the question: How do we get better?

6. CQI Process: PDSA Cycle The use of a deliberate and defined improvement process Plan-Do-Study-Act Cycle will be used for this project

7. CQI Process: PDSA Cycle When applying PDSA there are 3 key questions and 4 QI principles that will guide efforts. • The 3 key questions to keep front and center during QI work are: • What are we trying to accomplish? • How will we know that a change is an improvement? • What changes can we make that will results in improvement?

8. CQI Process: PDSA Cycle • The 4 CQI principles are: • Develop a strong client focus • Continually improve all processes • Involve stakeholders • Mobilize both data and team knowledge to improve decision-making

9. PDSA Cycle: Nine Steps The PDSA cycle is used to improve existing processes, not to plan or implement new programs Always relies on data to support its decisions

10. PDSA Cycle: Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 1-5) • Step 1: Identifying a problem • Step 2: Assembling a Q.I. team that can address the problem • Step 3: Identify what the true cause(s) of the problem may be, map the process to understand the existing activities that lead to the problematic issue • Step 4: Brainstorming and conducting background research to look for possible solutions • Step 5: Working to develop a theory of improvement that can be tested

11. Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 6) • Step 6: Test your theory of improvement • implement a small-scale test of the change to your process. • Collect, chart, and display data to determine the effectiveness of the improvement. • Document problems, unexpected observations, and unintended side effects, because these occurrences will aid in the learning process.

12. Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 7) • Step 7: Study the Results of your Test • Use data to determine if the test of your theory was successful • Compare the results of your test to the baseline data • Some questions to consider include: • Did your test work? How do you know? • Did the results match your theory/prediction? What do the data show? • Are there trends in your data? • Did you have unintended side effects? • Is there improvement? • Do you need to test the improvement under other conditions?

13. Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 8 and 9) • Step 8: Either standardize your improvement (if test was successful) or develop a new theory for improvement (if test was unsuccessful) • Step 9: Establish Future Plans • Sustaining change or beginning PDSA again

14. Tips for Beginning the Journey • CQI can be confusing, daunting - lots of jargon, different tools, methods, approaches (Google = 2,760,000 results) • Avoid starting with a large-scale project; think small and simple • Choose a QI effort related to a familiar process, program or area • Be patient – change takes time • Unanticipated results are not failures. They are learning opportunities for your next trial. We will be going through the journey together!

15. FOCUS First • Find an opportunity to improve. • Organize a team who understands the process. • Clarify the current knowledge of the process. In this stage, you are gathering the "who, what, when, and where“. • Understand the cause of process variation. Here, you ask yourself the "why" question. • Select the piece of the process you want to improve.

16. Self-Assessment Survey Assessed Knowledge, Attitudes and Practices (KAP) about CQI and Professional Development: • Role of CQI within Organization • Use of CQI Strategies and Techniques • Involvement in CQI • Use of CQI Techniques to examine Enrollment/Engagement • Desire to improve Enrollment/Engagement • Institutional supports in place to support a focus on Enrollment/Engagement • Understanding Capacity

17. Self-Assessment Survey • Conducted in November 2013 • Purpose: to capture what is known and unknown about the training process and the use of Continuous Quality Improvement (CQI) principles in five component areas: • professional development training • quality of professional development • role of continuous quality improvement in professional development activities • evaluation activities • data collection and analysis • 21 responses received

18. Summary of Survey Results

19. Summary of Survey Results Section 1: Professional Development Training • Members of the ECMHAB are involved in early childhood mental health initiatives, supportive of offering cross-disciplinary training and knowledgeable about the populations they serve: • 57% report having a full understanding of who needs PD • 76% work with other programs to enroll individuals in PD • All agreed that ECMH trainings should be offered across disciplines

20. Summary of Survey Results Section 1: Professional Development Training • Less is known about trainings themselves • 1/3 of ECMHAB members don’t know if the number of trainings meet the need, if trainings are well attended, if registration is easy to understand and complete, or if cost is a barrier • 76% of respondents disagreed that current communication methods adequately reach all individuals needing PD

21. Summary of Survey Results Section 1: Professional Development Training • There may be regional or specialty-specific deficits: • >half of respondents reported that the number of trainings is not sufficient to meet the need in their region or specialty. • Nearly half of respondents reported that participants do not know where to register and find training opportunities.

22. Summary of Survey Results Section 2: Quality of Professional Development • Feedback from survey was generally positive regarding the quality of current PD efforts. • Exceptions were: • Nearly ¼ of respondents disagreed (1) trainers are able to accommodate differing literacy levels and language barriers (2) early childhood programs are implementing evidence-based programs and practices

23. Summary of Survey Results Section 2: Quality of Professional Development • A significant number of individuals reported “Don’t Know” for several indicators of quality centering around two themes: • The demographics and values of the training participants • Their belief in evidence-based practices • The quality of the trainers themselves • Accommodating ethnic and cultural needs and differing literacy levels and language barriers

24. Summary of Survey Results Section 3: Role of CQI in PD Activities • Members of the ECMHAB are knowledgeable about both the DSS and ECCS project, reporting familiarity (95%) and confidence (80%) in both logic models. • 50/50 split between those with familiarity and experience in both CQI and PDSA and those without • 62% report needing training on CQI

25. Summary of Survey Results Section 4: Evaluation Activities • Most respondents are aware of the ongoing evaluation activities included in both the DSS and ECCS projects. • There is a significant percentage who report not knowing about evaluation activities: • Formally measuring participant knowledge and understanding after training (38%) • Evaluation plans (19%) • Collect quantitative and qualitative data to measure progress (24%)

26. Survey of Summary Results Section 5: Data Collection and Analysis Members are aware of the opportunity they have to provide input into quality improvement measures. A lack of awareness regarding what outcome data is available and collected for PD.

27. Next Steps • Provide ongoing training and education: • CQI • PDSA cycle • Specific information and updates on the data collected under the evaluation plan • PD qualifications for trainers 2. Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback

28. Focus for 2014 CQI Plan • Professional Development Training process • Qualifications of trainers • Demographics and values of training attendees (Who is attending?) • Types of data collected for each training

29. Focus for 2014 CQI Plan What does the Board feel needs to be the focus Suggestions follow based on the survey

30. Focus for 2014 CQI Plan 2. Quality • Cultural competence • Evidence based trainings • Stakeholder input

31. Focus for 2014 CQI Plan 3. Evaluation • Review of process and outcome measures • Monitoring of any factors that inhibit project implementation

32. Focus for 2014 CQI Plan 4. Performance Measure/ Outcomes - Review and monitoring of data collected following each training

33. Focus for 2014 CQI Plan • Investigate areas potentially in need of improvement: • Effectively reaching all individuals in need of PD • Communication methods and networks • Ensuring participants know where to register and how to find training opportunities • Understanding if, where, and/or what type of gaps exist in training