Laboratory QC and Patient Risk

1. Quality Control and Patient RiskCurtis A. Parvin, Ph.D.Manager of Advanced Statistical ResearchQuality Systems Division

2. Laboratory QC and Patient Risk • For the clinical laboratory patient risk is related to the reliability of the patient results they produce and report. • Laboratory Quality Control principles have been connected to the concept of patient risk for decades. • Specify the quality required of a patient result. • Determine the magnitude of “critical” out-of-control conditions with unacceptably high probability of producing patient results that fail to meet the required quality. • Design QC procedures with adequate power to detect “critical” out-of-control conditions. 2 |

3. Risk Management • A number of risk management guidelines have been published recently. • For manufacturers • ISO 14971: Medical devices – Application of risk management to medical devices • CLSI EP18: Risk management techniques to identify and control laboratory error sources • For the laboratory • ISO 22367: Medical laboratories – Reduction of error through risk management and continual improvement • CLSI EP23: Laboratory quality control based on risk management 3 |

4. The Risk Management Process • Risk Management provides a formal approach to • Identify potential failure modes in the lab • Rank the identified failure modes in terms of their risk • Establish policies and procedures to prevent or reduce (mitigate) the risks • Focus on the high ranked risks • This is generally a qualitative process • Risks are computed as a combination of • The probability of occurrence of harm • The severity of harm 4 |

5. Statistical QC and Risk Management • The question that we will address today is: How do statistical QC design principles that address questions such as • How many QCs should I examine? • What QC rule(s) should I use? • How frequently should I schedule QC evaluations? fit into the overall risk management process? 5 |

6. Risk Assessment (EP23 Figure) • Hazard Identification • Create a process map • Identify potential failures in each process step • Determine mechanisms in place to prevent or detect a failure • Risk Estimation • Assess the likelihood or probability of harm for each failure • Assess the severity of harm to a patient from each failure Risk Estimation Is the residual risk of harm clinically acceptable Risk Control What control processes are needed to lowerrisk to an acceptable level No Yes • The Laboratory’s QCP • Compile set of QC process into QCP • Review QCP for conformance to regulatory and accreditation requirements • Document and implement the set of control processes as the laboratory’s QCP 6 |

7. Risk Assessment (EP23 Figure) • Hazard Identification • Create a process map • Identify potential failures in each process step • Determine mechanisms in place to prevent or detect a failure • Risk Estimation • Assess the likelihood or probability of harm for each failure • Assess the severity of harm to a patient from each failure Risk Estimation Is the residual risk of harm clinically acceptable Risk Control What control processes are needed to lowerrisk to an acceptable level No Yes • The Laboratory’s QCP • Compile set of QC process into QCP • Review QCP for conformance to regulatory and accreditation requirements • Document and implement the set of control processes as the laboratory’s QCP 7 |

8. Probability of Harm Sequence of Events Creating Risk of Harm for a Patient Hazardous Situation P1 P2 P3 P4 P5 P6 Initiating cause Testing process failure Incorrect result generated Incorrect result reported Misdiagnosis Hazardous medical action Patient harmed EP23-A, Figure 6 ISO 14971, Figure H.1 ISO/TS 22367, Figure B.1 • Estimates of the probability of harm are desirable • For each potential mode of failure • Collectively for all possible failure modes 8 |

9. Estimating Probability of Harm P1: The probability that a testing process failure occurs P1 P1 P2 P3 P4 P5 P6 Initiating cause Testing process failure Incorrect result generated Incorrect result reported Misdiagnosis Hazardous medical action Patient harmed 9 |

10. P1: Probability of a Failure Mode • P1 = how likely is a given failure mode going to occur or more commonly • P1 can be the mean time between failures (MTBF) Information regarding MTBF may come from • Manufacturer • Historical failure data • Literature • MTBF can be converted to mean number of patient results between failures (MPBF) • MTBF = 30 days • 150 patient results examined per day (on average) • MPBF = 30 * 150 = 4500 patient results 10 |

11. Estimating Probability of Harm P2: The probability that a testing process failure generates incorrect patient results MPBF P1 P2 P2 P3 P4 P5 P6 MPBF Initiating cause Testing process failure Incorrect result generated Incorrect result reported Misdiagnosis Hazardous medical action Patient harmed 11 |

12. P2: Probability of Producing an Incorrect (Unreliable) Result • The number of unreliable patient results produced during the existence of an out-of-control condition depends on • The quality required for patient results • The type and magnitude of the out-of-control condition • The power to detect the out-of-control condition • When (how often) QC evaluations are performed • An out-of-control condition may result in many unreliable patient results being produced 12 |

13. Unreliable Patient Results ISO 15189 Clause 5.6.1: Laboratory QC should assure that patient results meet the quality required for their intended use The quality of a patient result depends on the difference between the correct value and the value reported. If the error in a patient’s result exceeds the allowable total error (TEa) The result is considered unreliable (or incorrect). It creates a hazardous situation for the patient. 13 |

14. Expected Number of Unreliable Patient Results Produced: E(NU) The number of unreliable patient results produced during an out-of-control condition (red asterisks) will depend on The magnitude of the out-of-control condition The power of the QC rule – Detection The number of patient specimens between QC events E(Nu) = Expected number of unreliable patient results generated during an out-of-control condition E(Nu) 14 |

15. Estimating Probability of Harm P3: The probability that incorrect results that are generated will be reported E(Nu) P2 P3 MPBF E(Nu) P3 P4 P5 P6 Initiating cause Testing process failure Incorrect result generated Incorrect result reported Misdiagnosis Hazardous medical action Patient harmed 15 |

16. P3: Probability of Reporting an Unreliable Patient Result • The probability of reporting an unreliable result that leads to an incorrect action depends on • The number of unreliable results produced because of an out-of-control condition • How and when results are reported • The likelihood of identifying and correcting a reported unreliable result before an incorrect action is taken 16 |

17. E(Nu) E(Nuf) E(Nuc) Expected Number of Unreliable Patient Results Reported • The patient results produced during an out-of-control condition can be divided into • Results prior to the last accepted QC event (Pre) • Results since the last accepted QC event (Post) • Bracketed QC: results aren’t reported • Immediate reporting: results should be repeated and updated in a timely fashion • E(Nuf) = Expected number of unreliable “final” results • E(Nuc) = Expected number of unreliable “correctable” results 17 |

18. Estimating Probability of Harm P4: The probability that an incorrect result leads to a misdiagnosis P5: The probability that a misdiagnosis leads to a hazardous medical action P6: The probability that a hazardous medical action leads to patient harm P4*P5*P6: The probability than an incorrect result leads to patient harm E(Nuf) P3 P4 P5 P6 MPBF E(Nu) E(Nuf) P4 P5 P6 Initiating cause Testing process failure Incorrect result generated Incorrect result reported Misdiagnosis Hazardous medical action Patient harmed 18 |

19. Probability of Patient Harm from an Unreliable Patient Result • The probabilities (P4, P5, P6) are associated with activities outside the laboratory • Their product equals the probability of patient harm given that an unreliable result has been reported Ph|u = P4*P5*P6 • Estimates of the probability of harm given an incorrect result are based on; • Medical literature • Medical judgment • Consultation with clinical colleagues 19 |

20. Estimating Probability of Harm P4 P5 Ph|u P6 MPBF E(Nu) E(Nuf) Initiating cause Testing process failure Incorrect result generated Incorrect result reported Misdiagnosis Hazardous medical action Patient harmed 20 |

21. Estimating Probability of Harm: Risk Management + Statistical QC Risk Management Statistical QC MPBF E(Nu) E(Nuf) Ph|u Initiating cause Testing process failure Incorrect result generated Incorrect result reported Patient harmed 21 |

22. Harmed Patients from Failure Mode Occurrence Probability of patient harm is The expected number of patients harmed due to incorrect results from a failure mode: Ph|u* E(Nuf) Divided by the average number of patient results examined between failure mode occurrences: MPBF E(Nuf) E(Nuc) E(Nu) Ph|u*E(Nuf) MPBF * * * Probability of Harm = Ph|u* E(Nuf) / MPBF 22 |

23. The quality requirement for the analyte • Allowable Total Error (TEa) • Test method performance • Instrument imprecision and bias • QC procedure • QC rule(s) used • Number of QC levels evaluated at each QC event • QC schedule (average number of patients examined for the analyte between QC events) • The quality requirement for the analyte • Allowable Total Error (TEa) • Test method performance • Instrument imprecision and bias • QC procedure • QC rule(s) used • Number of QC levels evaluated at each QC event • QC schedule (average number of patients examined for the analyte between QC events) Needed to compute the probability of an unreliable patient result Needed to compute the expected # of affected patient results What Info Do We Need to Predict E(Nu) and E(Nuf)? • The quality requirement for the analyte • Allowable Total Error (TEa) • Test method performance • Instrument imprecision and bias • QC procedure • QC rule(s) used • Number of QC levels evaluated at each QC event • QC schedule (average number of patients examined for the analyte between QC events) 23 |

24. Probability of an Unreliable Result Due to a Systematic Error Condition 24 |

25. Probability of an Unreliable Result Depends on TEa, Method CV and Bias 25 |

26. Expected # Affected Depends on QC Rule and QC Schedule 26 |

27. Expected # of Unreliable Patient Results 27 | 27 |

28. Ways to Reduce (or Manage) Patient Risk 28 |

29. Ways to Reduce (or Manage) Patient Risk • Manage the risk of unreliable patient results in the worst case • Limit the maximum values for E(Nuf) and E(Nuc) • Manage the overall expected risk of unreliable patient results • Limit the areas under the E(Nuf) and E(Nuc) curves • Manage the “defect rate” for unreliable patient results • Requires an estimate of the rate at which out-of-control conditions occur 29 |

30. Summary The laboratory has two main mechanisms to estimate and reduce the risk of patient harm: Identify as many potential failure modes as possible and seek ways to reduce occurrences of the identified failure modes Risk management Implement QC strategies that minimize the number of incorrect patient results that are reported when a failure mode does occur Statistical QC The combination enables a laboratory to address the full spectrum of patient risk implications for their operations 30 |

31. References • Parvin CA. Assessing the impact of the frequency of quality control testing on the quality of reported patient results. Clin Chem 2008;54:2049-54. • Parvin CA, Yundt-Pacheco J, Williams M. Analytical assessment in the clinical laboratory: assessing analytical quality goals when the same analyte can be tested on multiple systems is explored. Adv Admin Lab 2011;20(1):28-31. • Parvin CA, Yundt-Pacheco J, Williams M. The focus of laboratory quality control: why QC strategies should be designed around the patient, not the instrument. Adv Admin Lab 2011;20(3):48-9. • Parvin CA, Yundt-Pacheco J, Williams M. Designing a quality control strategy: in the modern laboratory there are 3 questions that must be answered. Adv Admin Lab 2011;20(5):53-4. • Parvin CA, Yundt-Pacheco J, Williams M. The frequency of quality control testing: QC testing by time or number of patient specimens and the implications for patient risk are explored. Adv Admin Lab 2011;20(7):66-9. • Parvin CA, Yundt-Pacheco J, Williams M. Recovering from an out-of-control condition: the laboratory must assess the impact and have a corrective action strategy. Adv Admin Lab 2011;20(11):42-4. • Parvin CA, Yundt-Pacheco J, Williams M. Sigma metrics, total error budgets, and quality control: Make sure your test system performance and quality control procedures are aligned with your quality goals. Adv Admin Lab 2012;21(1):40-4. 31 |

32. 32 |

33. 33 |

34. THANK YOU! 34 |