QC THE MULTIRULE INTERPRETATION

1. QCTHE MULTIRULE INTERPRETATION Chemometrics Department of Chemical Pathology, University of Pretoria, Dr R Delport 2003

2. Chemometrics Multirule QC uses a combination of decision criteria, or control rules, to decide whether an analytical run is in-control or out-of-control.

3. Chemometrics 13s refers to a control rule that is commonly used with a Levey-Jennings chart when the control limits are set as the mean plus 3s and the mean minus 3s. A run is rejected when a single control measurement exceeds the mean plus 3s or the mean minus 3s control limit.

4. Chemometrics 12s refers to the control rule that is commonly used with a Levey-Jennings chart when the control limits are set as the mean plus/minus 2s. In the original Westgard multirule QC procedure, this rule is used as a warning rule to trigger careful inspection of the control data by the following rejection rules.

5. Chemometrics 22s - reject when 2 consecutive control measurements exceed the same mean plus 2s or the same mean minus 2s control limit.

6. Chemometrics R4s - reject when 1 control measurement in a group exceeds the mean plus 2s and another exceeds the mean minus 2s.

7. Chemometrics 41s - reject when 4 consecutive control measurements exceed the same mean plus 1s or the same mean minus 1s control limit. 

8. Chemometrics 10x - reject when 10 consecutive control measurements fall on one side of the mean.

9. Chemometrics 8x - reject when 8 consecutive control measurements fall on one side of the mean.

10. Chemometrics 12x - reject when 12 consecutive control measurements fall on one side of the mean. The preceding control rules are usually used with N's of 2 or 4, which means they are appropriate when two different control materials are measured 1 or 2 times per material.

11. Chemometrics In situations where 3 different control materials are being analyzed 2of32s - reject when 2 out of 3 control measurements exceed the same mean plus 2s or mean minus 2s control limit;

12. Chemometrics 31s - reject when 3 consecutive control measurements exceed the same mean plus 1s or mean minus 1s control limit.

13. Chemometrics 6x - reject when 6 consecutive control measurements fall on one side of the mean. 

14. Chemometrics 9x - reject when 9 consecutive control measurements fall on one side of the mean. 

15. Chemometrics A related control rule that is sometimes used, particularly in Europe, looks for a "trend" where several control measurements in a row are increasing or decreasing: 7T - reject when seven control measurements trend in the same direction, i.e., get progressively higher or progressively lower.

16. Chemometrics • False alarms are minimized by using the 12s rule as a warning rule, then confirming any problems by application of more specific rules that have a low probability of false rejection (serial testing). • True alarms or error detection are maximized by selecting a combination of the rules most sensitive to detection of random and systematic errors, then rejecting a run if any one of these rules is violated (parallel testing). http://www.westgard.com/lesson3.htm QC - THE PLANNING PROCESS

17. Chemometrics The key in how to apply control rules with multiple materials and multiple runs is to identify which control results represent consecutive measurements; e.g., if one measurement is made on each of two different control materials in an analytical run, control rules can be applied as follows: • The two control results "within a run" can be inspected by applying a 13s rule to each material, as well as the 22s and R4s rules "across materials." • The 22s rule can also be applied to the last two measurements "within a material and across runs." • The 41s rule can be applied to the two control measurements in the current run and the two measurements in the previous run, i.e., the rule can be applied "across materials and across runs". • The 41s rule can also be applied to the last four measurements "within a material and across runs," which now requires the control results from the three previous runs. • The 10x rule can be applied to both control measurement in a run for the last five runs, or to the measurements on just one material for the last ten runs.

18. Chemometrics Because of these many possible applications of individual rules in a multirule QC procedure, it is best to provide specific directions for when to analyze controls, how to interpret the results, and what to do based on those results. • Statistical QC procedure. Use a 12s warning rule and the 13s/22s/R4s/41s/10x rejection rules with 2 control measurements per run. • Analysis of control materials. Analyze one sample of the Level A control and one sample of the Level B control in each run. • Interpretation of warning rule. If both control results are within 2s limits, report the patient test results. If one control result exceeds a 2s limit, inspect the control data as follows and reject the run if any control rule is violated

19. Chemometrics 4. Within run inspection of control results. Inspect the control results in the current run by applying the 13s rule to the results from each material and the 22s and R4s rules across materials. Note that the 41s and 10x control rules cannot be applied within a run because there are only two control measurements available. 5. Across run inspection of control results. Apply the 22s rule within each material across the last two runs; apply the 41s rule within each material across the last 4 runs; apply the 41s rule across the last two runs and the two measurements on each material; apply the 10x rule across the last five runs and the two measurements on each material. [Note that this protocol does not specify applying the 10x rule within each material across the last ten runs.]

20. Chemometrics 6. Interpretation of rejection rules. If none of the rules in steps 3 and 4 are violated, accept the run and report patient results. If any one of the rules in steps 3 and 4 is violated, the run is out-of-control; do not report patient test results. 7. Problem-solving. When a run is out-of-control, investigate the process and correct the problem, in the following way: • Determine the type of error occurring on the basis of the rule violated. Random error is usually indicated by the 13s or R4s rules, whereas systematic error is more likely indicated by the 22s,41s, or 10x rules. • Refer to trouble-shooting guides to identify possible causes for the type of error indicated by the control rule that was violated. • Inspect the testing process and identify the cause of the problem. • Correct the problem, then analyze control samples again to assess control status. • Repeat or verify the results on the patient samples once the method has been demonstrated to be in-control. • Consult a supervisor for any decision to report patient results when a run is out-of-control.

21. Chemometrics Example control results for this multiple rule application High: mean=250 and s=5) Low: mean=200 and s=4 Identify the rule at: 3,4,7,9,10, 11,12,14,20 http://www.westgard.com/lesson18.htm#terminology

22. Chemometrics Run 3 Both control results exceed their respective +2s limits, therefore there is a 22s rule violation across materials. A systematic error is most likely occurring and is affecting the results throughout the critical analytical range from at least 200 to 250 mg/dL.

23. Chemometrics Run 4 The high control result is below its -2s limit, which is a warning of a possible problem. Inspection with the 13s, 22s, and R4s rejection rules that can be applied within the run do not confirm a problem. Note that the across-runs rules would not be applied because the previous run was rejected.

24. Chemometrics Run 7 The high control result exceeds its +3s limit, therefore there is a 13s control rule violation. This most likely indicates random error.

25. Chemometrics Run 9 The high control result is below its -2s limit. Inspection of the control results by the rejection rules does not confirm a problem.

26. Chemometrics Run 10 The control chart for the high control material shows that the last two measurements have both exceeded the -2s limit, therefore a 22s rule violation has occurred within material and across runs. This situation would be consistent with a loss of linearity that is beginning to affect the high end of the analytical range.

27. Chemometrics Run 11 There is a 12s warning on the high level control material, but inspection doesn't show any other rule violations, therefore, the patient test results in this run can be reported.

28. Chemometrics Run 12 The control charts for the high and low materials show that the last four control observations have exeeded their respective +1s limits, therefore a 41s rule violation appears to have occured across materials and across runs.

29. Chemometrics Run 12 Note, however, that the QC protocol specified that a control result had to first exceed a 2s control limit before initiating the application of the 41s rule. Therefore, according to the protocol, this run would not be interpreted as out-of-control.

30. Chemometrics Run 14 The control results for the high material exceeds its +2s limit and the control result for the low material exceeds its -2s limit, therefore an R4s rule violation has occurred. This most likely indicates a random error.

31. Chemometrics Run 14 The control results for the high material exceeds its +2s limit and the control result for the low material exceeds its -2s limit, therefore an R4s rule violation has occurred. This most likely indicates a random error.

32. Chemometrics Run 20 The last five control results on the high material and the last five results on the low material all are lower than their respective means, giving a total of ten consecutive control results on one side of the mean. There is a 10x rule violation across runs and across materials, which indicates that a systematic error most likely has occurred.

33. Chemometrics QC - THE LEVEY-JENNINGS CONTROL CHART Exercise http://www.westgard.com/lesson12.htm Answer http://www.westgard.com/lssn12p2.htm