National Accreditation & QI: Try a Tool

1. National Accreditation & QI: Try a Tool Working with Data Julia Heany Michigan MLC-3

2. Data & QI • QI is a data driven approach to decision-making, but what does this mean? • We select and test improvements based on measurements and/or observations “If you can observe an event (or its effects) you can measure it. If you can measure it, you can improve it.” MI’s QI Guidebook, pg. 13

3. 3 Key Data Applications • Identifying opportunities for improvement • Do the data support or counter our understanding of the problem? • Establishing a baseline • Where are we now? • Testing a change • Was our change an improvement?

4. Data Roadblocks (and how to avoid ‘em) • Aim, theory, & data are out of alignment. • I have all these data, but they really don’t tell me what I need to know! • To Avoid: • Make sure your aim is SMART • Make sure your theory includes variables that you can measure or observe (is operationalized) • Make sure you have or can collect data that tell you something about the variables in your theory • QI is all about the practical – don’t collect data you don’t need.

5. Data Roadblocks (and how to avoid ‘em) • Pre-post data are used without attention to alternative explanations. • Immunizations went up [or down] this month! It worked [didn’t work]! • To Avoid: • Look for alternative explanations • Use multiple data sources and look for convergence • Look at data that are closely and logically related to your change • Use trend data (run & control charts) • QI provides simple, easy to use tools for understanding change over time.

6. Data Roadblocks (and how to avoid ‘em) • Good, helpful data are not used to their potential. • I have a stack of surveys, three reports from the State, and a spreadsheet full of numbers, but I don’t know what to do with them. • To avoid: • Make a data analysis plan when you’re planning your improvement. • Focus on the variables, not the numbers. • Look at your data. A picture is worth a thousand digits. • Which leads us directly into….

7. QI Data Tools • Schedule of Events • Check Sheet • Pareto Diagrams • Practice Exercise • Run Chart • Practice Exercise

8. Check Sheets: What’s going on?

9. Check Sheets: Purpose • To collect observational data • To organize existing data in a usable form • To help the team come to a common understanding of what they’re looking for and why they’re looking for it

10. Check Sheets: Example • When clients wait longer then 10 minutes to be seen, staff want to track the primary reason for the long wait • They think that long waits might happen because: • Lengthy appointments • Charts are disorganized and get lost • Multiple people are booked in the same time slot • Emergencies • Not enough staff • They think that there might be differences in numbers of and reasons for long waits between: • Clinics • Days of the week • Shifts

11. Check Sheets: Step by Step • Step 1: Define your variables

12. Check Sheets: Step by Step • Step 2: Design your check sheet, capturing all variables

13. Check Sheets: Step by Step • Step 3: Prep for Data Collection • Develop a timeline • Identify who will collect the data • Provide training • Step 4: Data Collection • Conduct quality checks

14. Check Sheets: Step by Step • Step 5: Summarize & Display the Data • Use Excel or other spreadsheet • Use Check Sheets themselves • Create summary stats & display using figures

15. Check Sheets: Step by Step • Step 6: Make sense of your results by examining your data • What causes occur most frequently? • Are there differences by observation time, date, or setting? • What can you change? What can’t you change? • Can this information help guide your improvement? • Can you use this information to help set reasonable goals for your improvement? • Can this information be collected over time as you make changes?

16. Check Sheets: Hints and Tips • Make sure everyone who is responsible for gathering data is on the same page before you get started. • Make sure your categories are inclusive and mutually exclusive • Think through how to handle situations that seem to have multiple causes • Look at your data in multiple ways.

17. Pareto Charts: What’s the problem?

18. Pareto Charts: Purpose • To identify the causes that are likely to have the greatest impact on the problem if addressed • “80% of the effects come from 20% of the causes” • To bring focus to a small number of potential causes • To guide the process of selecting improvements to test

19. Pareto Charts: Step by Step • Step 1: Identify potential causes of the problem you wish to study • Potential causes must be measurable or observable • Step 2: Develop a method for gathering your data • Historical data • Collection of new data • Check Sheets • Surveys

20. Pareto Charts: Step by Step • Step 3: Collect your data • Each time the problem occurs, make note of the primary cause • Returning to our previous example & using a check sheet to collect data…

21. Pareto Charts: Step by Step

22. Pareto Charts: Step by Step • Step 4: Order your results & calculate the percentage of incidents that fall into each category, for example:

23. Pareto Charts: Step by Step • Step 5: Display your data on a graph • The most commonly occurring cause should appear first, and the causes should appear in order • Word or Excel can be used, but paper and pencil work too • Label the x-axis (horizontal) with the causes, the left y-axis (vertical) with the percentage of occurrences with each cause, and the right y-axis with the cumulative percent. • Graph your data, for example:

24. Pareto Charts: Step by Step #of occurrences with each cause along y-axis Cumulative percent along y-axis Listed most to least common Causes along x-axis

25. Pareto Charts: Step by Step • Step 6: Make sense of your results by examining your data • Are a few causes driving the problem? • Can this information help you make decisions about the solution you want to try? • Does this information impact how you want to structure your aim statement or theory of change (if-then)? • Can you use this information to measure your results?

26. Pareto Charts: Hints and Tips • You’ll only learn about causes that you investigate - be inclusive! • Check and double check your data • Little errors can make a big difference • Results can be used in more than one way and they can be used differently at different points in time • Revisit your Pareto throughout your project – the meaning may change for you as you go

27. Genesee County Pareto

28. Practice Exercise

29. Run Charts: Do we see a trend?

30. Run Charts: Purpose • To study data measured over time • Run charts help to: • Measure the performance of a process • Identify trends over time • Measure change in outcomes following a change in process • Run charts can be used when you have, or can collect: • Quantitative or numeric data • On a measure of the performance of a process • Over time

31. Run Charts: When to Use • Example: • Each month the health department tracks the number of new BCCCP clients enrolled in the program to measure the impact of advertising the program in an additional local newspaper starting in April 2008.

32. Run Charts: Step by Step • Step 1 • Decide what data you need • Are the data you need already tracked? • Do they need to be collected? • Determine the timeframe & number of data collection points • Should you make your count annually, quarterly, monthly, weekly, daily, hourly? • Try to gather data from 20+ time points in order to establish a trend • Step 2 • Gather your data

33. Run Charts: Step by Step • Step 3 • Graph your data • On the Y-axis, set up a scale that corresponds with your measure • On the X-axis, set up a scale that corresponds with your measurement timeframe • Plot your data on the chart, placing one dot at each measurement point • Draw a line through your dots • Calculate the mean score and draw a line at the mean • Excel, Word, and other programs can help! • For example…

34. Run Charts: Step by Step Newspaper Publication Date

35. Run Charts: Step by Step • Step 4 • Make sense of your results by examining your data • Does the mean reflect an appropriate level of service or outcome of your process? • Is there a trend that should be investigated? • Do you see a shift in your data? Are there 8 or more consecutive points on one side of the center line? • Do you see a trend in your data? Are there six consecutive jumps in the same direction (up or down)? • Do you see a pattern in your data? Does a pattern recur eight or more times in a row? • Back to our example…

36. Run Charts: Hints and Tips • Every process will have some variation • Be cautious about assuming that variation from the average has meaning • Be sure to track data over a long enough period of time • This will help you identify the true mean and the true level of variability within the process

37. Practice Exercise