Understanding Variation in Your Charts

1. Understanding Variation in Your Charts Module 5

2. What is the state ofStatistical Control? • A stable mean over time with only random variation about that mean so that it is possible to predict how the system will behave in the near future. – a paraphrase of Shewhart’s 1931 definition. • No trends • No spikes • No steps • No cyclical patterns

3. Does this chart show data in statistical control?

4. How about this one?

5. Statistical Process Control vs. Stationary Time Series • Here SPC and Time Series Analysis agree. • A Time series is stationary if “there is no systematic change in the mean (no trend), if there is no systematic change in the variation, and if strictly periodic variations have been removed.” (Source: Chatfield (2004) The Analysis of Time Series: An Introduction.)

6. What is common cause variation? • The “faults” of the system that cause the random variation around the mean of a series that is in statistical control. • The causes that are common- they belong to the system. • “The natural variation inherent in a process on a regular basis.”* • “The variation expected to occur according to an underlying statistical distribution if parameters remain constant.”*

7. Special Causes of Variation • Causes that are assignable, foreign to the system, and special to a particular time period, group of workers, etc. • Special Causes create variation that is NOT random. They cause a process to be out of statistical control. • “Unnatural variation due to events, changes, or circumstances that have not previously been typical or inherent in the regular process.”* *Benneyan et al. Statistical process control as a tool for research and Healthcare improvement. Qual Saf Health Care 2003; 12:458-464

8. Can you tell whether IPC I had an effect?

9. Common Cause Variation Special Cause Variation Endemic Epidemic SPC vs. Epi

10. What is process capacity? • Not often used in health care. • The range a population distribution is able to stick within for 999 out of 1000 observations. • In a normally distributed process, this corresponds to plus or minus 3 sigma. • It is also the .00135 and .99865 quantiles of the probability plot, even if skewed and not normal.

11. Consider this: • If the process capability is between 10 and 50, but the specifications for the product are between 20 and 40, do we have a problem? • Related problem: If the GPRA measures goal is that 75 percent of the population needs to have a blood pressure below 130/80, what does the average blood pressure have to be? • Related problem: If the process capability is between 10 and 50, how far down does the value have to get following an intervention to detect change with a single point?

12. Conclusion regarding process capability • Background noise (process capability) determines whether your upper and lower specifications can be met. • With too much background noise, meeting a specification or goal where only the upper limit is specified may require the whole process to shift more than you want. Remember BP! • With too much background noise, can you see change in the process? When the standard error is 15, can you detect a change of 5?

14. If your current average systolic BP among persons w/diabetes is 140 and you want to meet a goal of 75%of patients with a systolic BP of 130 or less, what happens? 110 120 130 140

15. If you can do that without improving your process capability, the whole curve shifts. 110 120 130 140 And now your average blood pressure is 120, with half of your population less than 120. If you medicated people to that extent, is it safe?

16. Is Process Capability Important in Quality Measurement?

17. How do we detect special cause variation? Rules based on probability theory

18. Probability of saying there is a difference when there isn’t— Type I Error = α risk This curve is the probability distribution when there is a difference. This curve is the probability distribution when there is no difference. Probability of Saying there isn’t a difference where there is— Type II Error = β risk Statistical significance is defined here.

19. Rules for Interpreting Charts • Based on minimizing Type I error. • “Producer’s Risk” • “Provider’s Short Term Risk” • Based on minimizing Type II error. • “Consumer’s Risk” • “Patient’s Risk”

20. Rules for Run Charts(Run Charts with Median Centerline) • Too many or too few runs. See Test 1 on page 57 and the table on page 58 of Carey and Lloyd. • Run Length: A run with too many data points. • If < 20 observations, 7+ points in a run. • If 20+ observations, 8+ points in a run. • Trend: Consecutive increases or decreases. • If 5-8 obs, then 5 or more • If 9 to 20 obs, then 6 or more • If 20+, then 7 or more. Note: Rules preferred by Carey and Lloyd.

21. Rules for Control Charts

22. First, divide your control chart into zones. From Amin, 2001 Indian Health Service, DHHS

23. Rules for Control Charts(Nelson Rules) • One point beyond zone A. • 9 points in a row in zone C or beyond.* • Six points in a row (including endpoints) steadily increasing or decreasing. • 14 points in a row alternating up and down. • 2/3 points in Zone A or beyond. *Western Electric/AT&T rules require only 8.

24. Rules for Control Charts (Continued)(Nelson Rules) 6. 4/5 point in a row in zone A or beyond. 7. 15 points in a row in zone C. 8. 8 points in a row on both sides of the centerline with one in zone C.

25. Caution • Most of these rules assume a normal distribution. • With p, np, c, and u charts use only: • Test 1. One point beyond zone A. • Test 2. 9 points in a row in zone C and beyond* • Test 3. Six points in a row increase/decreasing. • Test 4. 14 points in a row alternating up and down. • The more tests you use, the greater the alpha risk, the risk of a Type I Error, which is?  • The more observations you have, the greater the alpha risk. *Only use this test if the distribution is symmetrical.

26. Your Turn

27. Your Turn Can you find another test met several times?