Developing and Managing Quality Improvement Learning Collaboratives

1. Developing and Managing Quality Improvement Learning Collaboratives September, 2010 Contributors: Brenda Joly PhD, MPH Maureen Booth, MRP George Shaler, MPH Ann Conway, PhD Lessons from the MLC States

2. Overview • Evaluation background • Components • Case study process, rationale, theory • Findings and opportunities • Planning and start-up • Managing the Learning Collaborative • Implications • Sponsors • Local health departments

3. MLC Evaluation • Quality improvement goal: • To promote the application of QI methods • Evaluation tools: • Annual survey • Mini-Collaborative survey • Quarterly reports • Case studies • Key informant interviews

4. Case Studies • Who? • Mini-collaborative members and organizers • LHD quality improvement teams • Why? • Multiple perspectives • “On-the-ground” understanding • What? • Site visits, interviews, document review • Observations of meetings

5. Literature-Based Theory

6. Planning and Start-Up

7. Key Finding #1: The Relevance of a Target Area is Critical and Impacts Engagement • Who selected targets? • State sponsors or statewide body • Local input • What criteria were use in selection? • Relevance and alignment with priorities • Need demonstrated by data

8. Key Finding #2: The Structure of a Mini-Collaborative Affects its Effectiveness • What works best? • Involving senior leadership is essential • Having diversity in faculty adds value • Having a sufficient size to promote exchange • Working on a consistent QI project • Having clear roles and responsibilities • Having prior experience working together

9. Key Finding #3: Defining Expectations & Communicating Them in Advance Builds Confidence • What did we learn? • Goals of mini-collaboratives varied and evolved • Build QI skills • Improve quality within target area • Goals should be aligned with timeframe and participant readiness • Expectations for QI projects should be focused, realistic and communicated

10. Key Finding #4: Advanced Planning Influences the Effectiveness of Learning Collaboratives • Who planned the mini-collaboratives? • State sponsors • Broader group of state, faculty and others • What should be planned? • Model, curriculum, workplan, use of faculty and evidence, tools to assess participant knowledge and measures to monitor progress

11. Key Finding #5: The Level of Effort Among Sponsors and LHDs is Often Underestimated • What did we learn? • The planning and start-up phase is especially time consuming for sponsors • Having outside faculty helps • Level of effort is often underestimated if expectations are unclear or scope is unrealistic • Competing priorities are a reality

12. Managing the Learning Collaborative

13. Key Finding #1: Opportunities for Timely and Frequent Application of QI Tools are Essential • What was most helpful? • Learning/using fewer tools • Training on when and how to use • Providing opportunity for immediate application • Providing feedback on use • Giving LHDs a chance to practice, practice, practice

14. Key Finding #2: Communication With and Among Members Enhances Learning Experience • What was most helpful? • Opportunities for ongoing information exchange • In-person meetings • Structured learning sessions • Communicating the value of QI to LHDs • Site visits to LHDs • Communicating clear expectations

15. Key Finding #3: Several Factors May Strengthen or Impede a QI Learning Collaborative • Facilitating factors? • Commitment to data driven QI and accreditation • Leadership buy-in • Impeding factors? • Concurrent QI skill development at state and local level • Lack of focus and clear expectations, unrealistic goals, turnover, and competing priorities

16. Key Finding #4: It is Critical for Participants to Have Access to Adequate Resources • Technical assistance: • There should be a good mix of content expertise and QI expertise among sponsors and faculty • Other resources: • Evidence needed to select QI intervention should be made available • Financial resources may boost efforts of LHDs

17. Implications for Sponsors • Some considerations… • Set time aside for advance planning • Recruit credible and skilled faculty • Assess QI knowledge beforehand • Recruit the enough and the right mix of participants • Embed the application of QI into the process • Communicate with the group frequently • Follow a structured approach

18. Implications for LHDs • Some considerations… • Garner support of leadership and staff • Promote the value of QI • Align QI project with job responsibilities • Actively engage in information exchange • Use faculty and available technical assistance • Remain open to the process and outcomes • Document your efforts • Share your findings

19. Developing and Managing a Quality Improvement Learning Collaborative September 2010 Contributors: Cathy Montgomery, M.S., ASQ-CQIA Sandra Ruzycki, M.P.H., Quad-R Baker, Clay, DeSoto, Duval, Glades, Martin, Nassau, St. Johns and St. Lucie County Health Departments Lessons from Florida

20. Planning and Start-Up

21. Plan, Plan, Plan • Use data to determine target area • Overweight and obesity among children 6-19 • Create partnerships • Outline timeframes and expectations Ψ • Establish criteria for selecting participants Ψ = lesson learned

22. Use Criteria

23. Childhood Obesity Mini-Collaborative

24. Establish (facilitator) Goals • Use of quality improvement tools and methods • Support participants in their projects • Communicate project impacts and outcomes • Local, state, national venues

25. Select an Improvement Model • Quality Improvement Control Story Ψ • Founded on PDCA • Integrates QI tools and methods throughout the process

26. Training and Tools • Assess participant’s knowledge of QI tools Ψ • Provide relevant training • Just-in-time • Face-to-face Ψ • Allow participants to practice using tools and process

27. Results: Training • Participation in QI training improved overall knowledge of QI tools – 84% • Baseline = 84% • Confident in their ability to use QI tools – 68% • Baseline = 62% • % of participants that rated level of use of QI tools after training as: moderate – 58% • Baseline = 62% high – 21% • Baseline = 15% *Surveys conducted March 2009 (baseline) and February 2010

28. Results: Tools % of participants that indicated a high readiness to: • Use QI tools in other initiatives – 83% (n = 15) • Incorporate QI tools into ongoing work – 67% (n=12) • Development of evaluation measures – 56% (n = 10) • Present QI tools to CHD staff – 50% (n=8) Ψ

29. Technical Assistance • Conduct coaching calls • Using quality improvement process and tools • Creating action plans and storyboards • Developing methods and measures for evaluation • Analyzing data results • Resources • Subject matter expertise • Books, articles, and research related to QI and projects • Evaluation database

30. Evaluation Database

31. Managing the Learning Collaborative

32. Collaborative Teams • Counties created teams consisting of internal staff • 65% of participants indicated multiple CHD programs were represented on team • 3 of 9 counties had QI program staff on the team

33. Results: Collaborative Teams • Team meetings were random and unproductive Ψ • 53% of participants stated team meetings were only slightly productive • 53% moderately or slightly agreed there was consensus on project goals and procedures • % of participants that strongly or moderately agreed project team members: • Got along well – 94% • Treated each other fairly – 71% • Had necessary skills for this project – 65%

34. Results: CHD Support 59% rated level of CHD support was very or somewhat supportive 71% indicated staff time and resources were fully, mostly or somewhat available

35. Communication • Provide venues for participants to share ideas • Monthly conference calls/web ex • Face-to-face meetings • SharePoint site • Conduct one-on-one coaching calls • Communicate progress of collaborative projects Ψ

36. Results: Communication • % of participants who rated the following resources as useful • One-on-one coaching calls: 65% • Monthly conference calls: 65% • SharePoint site: 59% • Collaborating with other MCLC counties: 71% • Collaborating with other agencies: 53% Ψ • % of participants who agreed or strongly agreed HPI provided: • Effective communication: 94% • Effective project coordination: 95%

37. Overall Experience • 76% rated their overall experience as excellent or good • 94% rated the level of success of their project as excellent or good • 88% indicated they were likely to participate on another learning collaborative • 76% reported they were likely to implement a new QI project in their CHD

38. Lessons Learned Ψ What else did we learn? • Planning and implementation is time consuming • 41.2% of participants spend 1-5 hours per week on project; others indicated they spend more time! • Don’t assume what they know • Competing priorities are a reality