Concepts in Quality Control Data Management

1. Concepts in Quality Control Data Management Thomasine Newby BS, MT (ASCP) Bio-Rad Laboratories Quality System Specialist Field Marketing

2. Objectives • At the conclusion of this program, participant • should be able to: • Understand how to plan statistical Quality Control • Learn how to derive QC Statistics • Evaluate QC using Westgard rules and graphs •  Discuss the types of errors found when evaluating QC and the resolution of these errors.

3. Reference Material Clinical and Laboratory Standards Institute •  Statistical Quality Control for Quantitative Measurement Procedures: Principles and Definitions; Approved • Guideline – Third Edition (C24-S3, Vol. 26 No. 25)

8. Quality • Laboratory Quality means achieving positive patient outcomes through control of pre-analytical, analytical, and • post-analytical error.

9. Pre-Analytical Errors •  Sample taken from wrong patient •  Wrong sample analyzed •  Incorrect test ordered/performed •  Patient sample not properly preserved •  Patient sample not properly collected – IV arm •  Incorrect or incomplete patient instruction and preparation

10. Post-Analytical Errors •  Wrong result reported or charted – transcription errors •  Computer error – LIS integrity •  Incorrect interpretation of test results provided to clinician •  Incorrect patient normal ranges • Results reported from an instrument not used to establish the normal range

11. Analytical Errors •  Instrument malfunction •  Lack of maintenance •  Inadequate environmental control •  Inadequate electrical supply •  Inadequate water supply •  Inadequate operator training •  Incorrect calibration •  Use of compromised reagents and/or calibrators

12. First Step Selecting a Quality Control Product

13. Selecting Control Materials •  Menu •  Shelf life/Open-vial Stability •  Vial to vial variability •  Medical relevant levels •  Challenge range of instrument •  Matrix/preservatives •  QC material different from calibrator material •  Number of levels and concentration – sufficient to determine proper method performance over the measuring range of interest.

14. Second Step QC Strategies

15. QC Strategy • Define QC material • Decide the number of measurements and the location of QC • Set the control limits • Define Error – random/systematic • Decide on the QC rules for acceptable run •  Outline the response to the data acceptance decision (out-of control)

16. Frequency and Location •  Frequency of QC - QC must be analyzed once during a run • Run is defined as interval that the method is considered stable •  Location of QC - QC results evaluated before reported patients - Placement immediately after calibration may give false low imprecision and shift during run

17. Set Control Limits Establishing Data Parameters

18. Establishing Data Parameters • Determination of valid mean and standard deviation values are crucial to successful data rejection or acceptance by error detection schemes.

19. Establishing Data Parameters • Parallel Testing vs. Product Inserts

20. CLIA Regulations • “The stated values of an assayed control material may be used as the target values provided the stated values correspond to the methodology and instrumentation employed by the laboratory and are verified by the laboratory.” • CLIA 88, Final Rule • Federal Register, February 2, 1992 • Standard 493.1218

21. CLSI Recommendations • “If assayed controls are used, the values stated on the assay sheets provided by the manufacturer should be used as guides in setting the initial control limits for testing new materials. Actual values for the mean and standard deviation must be established by serial testing in the laboratory. The observed mean should fall within the range published by the manufacturer.” • Clinical and Laboratory Standards Institute • Volume 26 Number 25 • 2006

22. QC Statistics • The first step in defining performance limits • ► Mean. . . . . . . . . . . . . . . . . . . x • ► Standard Deviation . . . . . . . .s • ► Coefficient of variation . . . . . cv

23. Mean • Arithmetic average of a set of data points •  Sum of values divided by the number of values • a + b + c ……+ n • n

24. Standard Deviation • A measure of variability of the data points •  The degree of dispersion around the mean • SD = the positive square root of the • population variance

25. Coefficient of Variation, CV • Measure of variability – random or systematic •  Expressed as a percent • CV = SD/Mean X 100

26. Calculating A Mean/Standard Deviation • CLSI Recommendations •  Standard •  Provisional

27. Things to Consider When Calculating QC Statistics •  Do I have a sufficient number of data points for the calculation? • Have I used a sufficient number of analytical runs to account for random variables? -Multiple calibrations - Multiple reagent lots - Maintenance - Technologists - Environmental conditions

28. Things to Consider When Calculating QC Statistics • Did I follow “normal” procedure each analytical run? •  Have I appropriately excluded outliers?

29. Test for Outliers •  Suspect low value – arrange values from high to low value •  Suspect high value – arrange values from low to high •  Refer to table – “Significance limits for testing extreme values of a sample” •  Column – use 0.05 (5% of values may be outside 2sd) •  Row – use the number of data points •  Calculate value – compare to table •  Calculated value > table value – Reject •  Calculated value < table value – Accept

30. Table Value

31. M.G. Natrella, Experimental Statistics, 91, 1963, NIST

32. Calculating A Mean/Standard Deviation • NCCLS Recommendations •  Standard •  Provisional • 95% Confidence Interval

33. Estimation of a 2s Range • Estimation of a 2s Range for a small sample by calculating a 95% confidence interval. • SE=s/ n SE=Standard Error of the Mean • n = 10 n = 5 • x = 398.2 x = 401.2 • s = 5.41 s = 4.97 • SE = 1.71 SE = 2.22 • LL = 398.2 – (1.71)(2.262) LL = 401.2 – (2.22)(2.776) • UL = 398.2 + (1.71)(2.262) LL = 401.2 + (2.22)(2.776) • 2s Range (est.) = 394.3-402.1 2s Range (est.) = 395.0-407.4

35. Standard Deviation Index, SDI • CALCULATION: • SDI= [Lab Mean – Peer Group Mean] • Peer Group’s 1 standard deviation • Use to assess bias compared to a peer group • TARGET SDI IS 0.0, INDICATING THAT THE LAB’S • MEAN VALUE IS THE SAME AS THE PEER’S.

36. Coefficient of VariationRatio, CVR • Calculation • Lab’s Monthly CV • Peer Monthly CV IDEALLY, CVR ≤ 1.0, SINCE YOUR VALUES ARE FROM A SINGLE LAB, WHILE THE PEER CV IS FROM SEVERAL. IF CVR = 1.5 to 2.0, THE LAB IS 50-100% LESS PRECISE THAN IT’S PEER GROUP, USUALLY REQUIRING INVESTIGATION.

37. Frequency Of Occurrence

38. Define Errors Random/Systematic

39. Types of Analytical Errors •  Random • - Precision • - Inherent to the test system • - Always present • - Minimize any increase • Systematic - Accuracy - Shifts - Trends

41. Set Quality Control RulesWestgard Rules

42. Westgard Rules Basics • Error Detection – 100 % • Identify the source of error • Random • Systematic • False Rejection – 0% • Multi-rule improves error detection with a low probability • of false rejection • Minimum of 2 controls per run • Not all rules are necessary for all tests

43. Westgard Rules Error Detection •  Random Error – imprecision • 1 2s, 1 3s, R4s •  Systematic Error – bias • 1 3s, 2 2s, 4 1s, 10x