QUALITY ASSURANCE OF TESTING OPERATIONS

1. QUALITY ASSURANCE OF TESTING OPERATIONS Thomas L. Hearn, Ph.D. Deputy Director Division of Laboratory Systems Public Health Practice Program Office

2. Quality Assurance “Planned and systematic activities to provide adequate confidence that requirements for quality will be met” (ISO 8042, 3.4; NCCLS)

3. Class Exercise: Describe the Steps in a “Routine” Process

4. T h e W a l l S t r e e t J o u r n a l Monday: February 2, 1987 “Medical Labs, Trusted as Largely Being Error -Free, Are Far From Infallible”

6. Benefits of Laboratory Quality Assurance • QA provides evidence of good performance • Laboratory mistakes are prevented • Significant improvements in testing performance can be achieved* *Research findings from the College of American Pathologists, the University of Wisconsin, and the Centers for Disease Control and Prevention

7. Why Is Quality Assurance of Testing Important ? • Public expects high quality • Defines parameters & quality goals • Evaluation & improvement system • Assures reliability & comparability of results • Cost effective • Even the simplest of testing is not foolproof

8. Questions to Ask In Assessing a National QA System • How is testing organized in-country? • Who makes testing policies? • Is there any central oversight? • Are laboratories required to have QA programs? • Which international agencies are, or were, active in technical assistance

9. Questions to Ask In Assessing a National QA System • How are decisions made about which tests can be used? • At the national level, who has authority & responsibility for QA? • What is the role of laboratory directors (national, provincial, and district) in QA? • How is training provided?

10. Quality Assurance “Planned and systematic activities to provide adequate confidence that requirements for quality will be met” (ISO 8042, 3.4; NCCLS)

11. Testing Process Analytic Specimen processing Specimen storage Reagent preparation Equipment checks Test performance Post-Analytic Results review Quality control review Report test results Results interpretation Pre-Analytic Test selection Patient instructions Test requisition Collect, label, transport specimen

12. Q A Quality Assurance Model U.S. Laboratory Regulations STAFF Q P C T

13. Quality Assurance (QA) • Staffing / personnel • Quality control (QC) • Proficiency testing (PT) aka External quality assurance (EQA)

14. Staffing Components • Able to execute responsibilities of director, supervisor, testing personnel • Appropriate educational credentials • Appropriate experience • Receive training • Competency assessments

15. Quality Control of theTesting Process Analytic Specimen processing Specimen storage Reagent preparation Equipment checks Test performance Post-Analytic Results review Quality control review Report test results Results interpretation Pre-Analytic Test selection Patient instructions Test requisition Collect, label, transport specimen Safety

16. Pre-Analytic QC Components • Safety • Specimen collection and handling • Test requisition and records • Referral for testing

17. Analytic QC Components • Facilities • Test methods, equipments, reagents, supplies • Validation of test methods • Equipment checks • QC monitoring procedures • Procedure manual

18. Analytic QC Components • Facilities • Test methods, equipments, reagents, supplies • Procedure manual • Validation of test methods • Equipment checks • QC monitoring procedures

19. Sensitivity, Specificity, and Prevalence If, prevalence = 20% sensitivity = 99.9% specificity = 99.9% Then, predictive value of positive = 99.6% predictive value of a negative = 100% If, prevalence = 20% sensitivity = 80% specificity = 80% Then, predictive value of positive = 50% predictive value of a negative = 94.1%

20. Sensitivity, Specificity, and Prevalence If, prevalence = 0.15% sensitivity = 95% specificity = 95% Then, predictive value of positive = 2.8% predictive value of a negative = 100% If, prevalence = 0.15% sensitivity = 99.9% specificity = 99.9% Then, predictive value of positive = 60% predictive value of a negative = 100%

21. Analytic QC Components • Facilities • Test methods, equipments, reagents, supplies • Procedure manual • Validation of test methods • Equipment checks • QC monitoring procedures

23. Using Quality ControlReduces Error Rate Sample HIV Number ------------Errors------------- Ab ReactivityCorrect With QC Without QC p-value Negative 6168 3 19 0.014 Strong 4896 4 3 0.830 Positive Weak 7021 152 307 0.0067 Positive -------------------------------------------------------------------------------------------------------------------- Total 18085 159 329 0.0023 (97.37%) (2.20%) ( 2.90%) Source: Model Performance Evaluation Program, CDC

24. Post-Analytic QC Components • Validation of test results • Reporting of test results • Interfacing to Laboratory Information Systems (LIS’s) • Corrective action plan • Records maintenance

25. Quality Assurance (QA) • Staffing / Personnel • Quality control (QC) • Proficiency Testing / External Quality Assurance

26. Proficiency Testing or External Quality Assessment • Central organization sends out challenge specimens for testing, laboratories’ results evaluated, laboratories sent scores • Split specimens exchanged with referral laboratories • Alternative methods

27. External Quality Assurance • Early warning-system for problems • Measure of laboratory quality • Valuable benchmarking tool (standardization and traceability) • Indicator of where to direct improvement efforts • Monitor of changes in technology and testing practices (evaluation component)

28. Activities Linking QA Components • Assess personnel competencies • Assess pre-analytic, analytic, and post-analytic QC system • Assess PT/EQA performance • Implement communication systems for customers and staff, and systems for complaints investigations

29. Sources of Laboratory QA Guidance and Information • World Health Organization (WHO) • UNAIDS • International Standards Organization (ISO) • NCCLS • CDC R&R’s • APHL recommendations • Professional & accrediting organizations • United States laboratory regulations

30. If even the most conservative estimates of gains in improved testing quality are used, thousands of testing mistakes may be prevented yearly by laboratory quality assurance programs.