1 / 31

Effective Control of the Examination Process

Effective Control of the Examination Process. “Reunión de Expertos” 5 Ciclo Internacional De Conferencias De La Calidad. PARTICIPANTS Laura Mercapide, Argentina Amadeo Saez, Brasil Aída Porras, Colombia Oscar Martínez, Colombia Enrique Amaya, Perú Margarita Iturriza, Venezuela

ulmer
Download Presentation

Effective Control of the Examination Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Effective Control of theExamination Process “Reunión de Expertos” 5 Ciclo Internacional De Conferencias De La Calidad

  2. PARTICIPANTS Laura Mercapide, Argentina Amadeo Saez, Brasil Aída Porras, Colombia Oscar Martínez, Colombia Enrique Amaya, Perú Margarita Iturriza, Venezuela Erik Mendoza, México Eduardo Brambila, México Arturo Terrés, México Coordinator James Westgard, United States

  3. Our Focus is on the Examination Process • Pre-analytic and post-analytic errors are also of concern, but our charge is to focus on the analytical part of the examination process • Evidence in scientific literature indicates analytical errors are still major source of problems leading to mistreatment and harm to patients

  4. What errors have been observed in the Total Testing Process? Plebani & Carraro. Clin Chem 2007:53:1338-42 60% 15% 25% Pre-analytic Analytic Post-analytic • Patient preparation • Specimen acquisition • Specimen processing • Sample transport • Physician test order • Sample aliquot • Analyzer setup • Test calibration • Quality control • Reportable test • Test report • Transmittal of report • Receipt of report • Review of test results • Action on test results 5

  5. Lab Errors and Patient Care Plebani-Carraro study ClinChem 2007:53:1338-42 51,746 tests 393 questionable results 160 confirmed as laboratory errors 46 caused inappropriate patient care 24 of those were analytical errors Analytical errors are still major cause (over 50% cases) of inappropriate patient care 6

  6. Need for Guidance on “Effective Control of Examination Process” for Latin American Countries • Important to be relevant and practical for local laboratories • Must consider national and governmental interests and requirements • Must consider professional assessment of needs and practice guidelines

  7. General guidance on “effective control of examination process”? • ISO 15189: Medical Laboratories – Particular requirements for quality and competence • ISO 15198: Validation of user quality control procedures by the manufacturer • CLSI C24-A3: Statistical Quality Control for Quantitative Measuring Processes – Principles and Definitions • CLSI EP23-P: Laboratory Quality Control based on Risk Management

  8. ISO 15189 Guidance for “Assuring Quality” of Examination Process • 5.6.1. “The laboratory shall design internal quality control systems that verify the attainment of the intended quality of results…” • Medical relevance of laboratory tests is an important consideration! • Comparability of test results is important for medical relevance! • How define medical relevance?

  9. ISO 15189 “Assuring quality” • 5.6.2 ..determine uncertainty of results, where relevant and possible • 5.6.3 …ensure that results are traceable • 5.6.4 …participate in interlaboratory comparisons • 5.6.5 …if EQA not available, develop a mechanism for determining acceptability • 5.6.6 For examinations performed using different procedures or at different sites, define a mechanism for verifying comparability of results

  10. Particular issues assigned to this work group • What frequency of QC is sufficient? • How important are recommendations from manufacturers for QC? Should laboratory modify recommendations? • How often for EQC or PT?

  11. What frequency of QC? • CLSI guidance on defining run length • “An analytical run is an interval (i.e., a period of time or series of measurements) within which the accuracy and precision of the measuring system is expected to be stable; between which events may occur causing the measurement process to be more susceptible (i.e., greater risk) to errors that are important to detect.”

  12. Factors Affecting Run Length (1)Events and non-events Event driven QC Known, scheduled and expected changes Non-event driven QC Other things that happen Parvin’s concepts Ref: Parvin, Gronowski. Effect of analytical run length on QC performance and the QC planning process. Clin Chem 1997;11:2149-2154. 13

  13. Factors Affecting Run Length (2)Mode of Operation Batch mode All patient specimens and control samples are analyzed together Patient results not reported until control results are validated Continuous mode Patient results are being reported as they are determined Control samples are analyzed periodically 14

  14. Factors affecting Run LengthCost-Effectiveness • Number of controls relative to number of patient specimens • Levels of controls

  15. Factors Affecting Run Length (3)Strategies Manufacturer’s instructions may provide minimum strategy – e.g., 2 levels per day + Event QC to assess significance of changes in the testing process + non-event QC to monitor process during routine operation Multi-stage QC for “startup,” “monitoring,” and “patient data QC” 16

  16. Factors Affecting Run Length (4)Strategies Sigma QC – relative amount of QC can be related to method’s sigma-performance Risk analysis and residual risks – guidance for susceptibility testing Consensus of experts – professional practice standards Experienced judgment – knowledgeable analyst has expertise about stability and susceptibility of testing process 17

  17. Events Operations Continuous mode Calibration, reagents Stats Maintenance System stability LIS Operators Components Susceptibility Batch mode Lab conditions Parts Unexpected events Batch size Analyte stability Reporting interval Residual risks Cost of repeat analysis Models Strategies Empirical Manuf. recommendations Statistical Measures Batch processing Sigma Assumptions Economic Event+Monitor Risk analysis Error frequency, f QC performance goals Residual risks Q/P Event only Consensus of experts Residual risks Experienced judgment Run length and frequency of controls 18

  18. What frequency of QC is sufficient?Recommendations (1) • Strategy • Define length of run • In terms of time, numbers of patient samples, mode of operation • Importance of “events” or changes that occur with the process that require verification by controls • Medically important concentrations for controls • General practice to use two levels of controls • Sometimes advisable to have three levels • Many factors to consider to optimize run length or frequency of QC

  19. What frequency of QC? Recommendations (2) • For small runs, utilize “batch” strategy • QC at beginning • QC at end • Release results after inspect all controls and reviewing patient results when necessary

  20. What frequency of QC?Recommendations (3) • For large runs, highly automated systems with continuous reporting of results • Controls at beginning of run • Right QC design to detect medically important errors • + Controls for events • e.g., Change of reagent lots • + Controls to monitor performance during run • Or, possibly use mean or median of patient data to monitor stability during run • + Controls at end of run

  21. How important are the manufacturer’s QC directions?Recommendations (4) • Provide minimum requirements that the laboratory must satisfy • E.g., calibration, preventive maintenance, etc. • Laboratory is still responsible for design of IQC system • Intended clinical use • Observed method performance • Necessary QC rules and Numbers of measurements

  22. How important manufacturers QC directions? Recommendations (5) • Need for “independent” control • “Third party control” • Traceability is an important responsibility of manufacturer • Calibration materials and process • Verification/validation of method performance is an important responsibility of the laboratory • EQA/PT important responsibility in monitoring/measuring accuracy over time • “Commutability” important characteristic of materials

  23. How often EQA/PT?Recommendations (6) • At least monthly • With fast turnaround of results to be useful for identifying bias and making improvements in the laboratory • Approved EQA program preferred • ILAC G13:08/2007 • ISO 17043 • Most essential information – bias observed vs “assigned value”

  24. Other issues of interest • Medical relevance • “Intended use,” “intended quality of test results” • Traceability • Comparability of test results • Validation of method performance • Design of IQC • Available planning approach, tools

  25. How assure quality? (1a) Regulatory & Accreditation Requirements • Define Goals for • Intended Use • (TEa, Dint) (1b) Clinical and Medical Applications (2a) Traceability & Calibration (2) Select Analytic Measurement Procedure (2b) Manufacturer’s Reference Methods & Materials (3) Validate Method Performance (CV,bias) (3a) Manufacturer’s Claims (4) Design SQC (rules, N, F) (5a) Manufacturer’s Risk Analysis (5) Formulate AQC Strategy (5b) Lab Evaluation of Residual Risk (11) Improve AQC Effectiveness (10) Monitor AQC Effectiveness (f), EQA (6) Develop AQC Plan (6a) QC Toolbox (9) Measure Quality & Uncertainty (7) Implement AQC System 26 (8) Verify Attainment of Intended Quality of Test Results

  26. ISO 15198 Validation of QC Procedures • QC procedures shall be validated to assure that failures are not a hazard to patients • Recommends use of risk analysis • Conventional statistical quality control procedures (e.g., as described in CLSI C24) are considered adequate • Validation may be based on simulated effects of errors on performance data

  27. Define quality specifications for test Select appropriate control materials Determine method performance Identify quality control strategies Predict QC performance Specify goals for QC performance Select QC to satisfy goals CLSI C24 QC Planning Process Calculate Sigma %TEa-%Bias %CV Utilize Sigma-metric QC Selection Tool

  28. Sigma Scale Desirable Error Detection Desirable False Rejection Probability for Rejection (P) Systematic Error (SE, multiples of s) Sigma-metrics QC Selection Tool2 Levels Control 3s 4s 5s

  29. EP22, EP23 on Risk Analysis • Original purpose of CLSI project was to develop scientific approach for defining frequency of QC • Adopted “risk analysis” approach • Failure-modes and risk should provide guidance on need for control mechanisms and frequency of QC • Analytical QC Plan should be the outcome of the risk analysis process

  30. Important Considerations in Future QC Systems • Design • Is there a scientific basis and approach for selecting parameters and setting limits on basis of intended quality of results? • Validation • Is there an objective approach for assessing the reliability of technical and medical decisions on control status? • Control • Is there a quantitative process for monitoring and verifying the attainment of intended quality of test results?

More Related