1 / 27

On the Role of Proficiency Testing Schemes in Laboratory Accreditation

On the Role of Proficiency Testing Schemes in Laboratory Accreditation. ISO/IEC 17025 and PROFICIENCY TESTING. 5.9 Assuring the quality of test and calibration results

hao
Download Presentation

On the Role of Proficiency Testing Schemes in Laboratory Accreditation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On the Role of Proficiency Testing Schemes in Laboratory Accreditation

  2. ISO/IEC 17025 and PROFICIENCY TESTING 5.9 Assuring the quality of test and calibration results The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following: a) regular use of certified reference materials and/or internal quality control using secondary reference materials; b) participation in interlaboratory comparison or proficiency-testing programmes; c) replicate tests or calibrations using the same or different methods; d) retesting or recalibration of retained items; e) correlation of results for different characteristics of an item. NOTE The selected methods should be appropriate for the type and volume of the work undertaken.

  3. EXTERNAL QUALITY CONTROL: PROFICIENCY TESTING Inter-laboratory comparison Target value and standard deviation based upon results of participants Proficiency testing Organizer decides on target value and target standard deviation

  4. PROFICIENCY TESTING, WHY: The prime purpose is to allow participating laboratories to become aware of unsuspected errors in their work and to take remedial action. The participant can make three comparisons of performance: - with an externally determined standard of accuracy- with that of peer laboratories- with its own past performance

  5. ISO/IEC 17025 and PROFICIENCY TESTING PLEASE NOTE: Interlaboratory comparisons may be designed for purposes other than proficiency testing: - validation of methods- characterization of candidate reference materials- self-assessment of a laboratory in a performance test A precondition for selection is that the scheme is operated according to ISO/IEC Guide 43 and/or ILAC Guide G-13.

  6. ISO/IEC 17025 and PROFICIENCY TESTING Why mandatory PT activities ? 1) ILAC requires a policy of the Accreditation Bodies related to the participation of accredited laboratories in PT. 2) ILAC makes recommendations 3) A National Accreditation Body may transfer this recommendation into its own policy, e.g.: …one PT activity prior to gaining accreditation and one activity relating to each major sub-area of major disciplines of a laboratory’s scope of accreditation at least every four years; where applicable and where practical….

  7. ISO/IEC 17025 and PROFICIENCY TESTING Does any PT-scheme exist for the tests and samples normally analyzed by the laboratory ? Sources: - National accreditation bodies - Peer laboratories already participating - PT-organizers in the own country - Internet e.g. www.eptis.bam.de - http://www.ianz.govt.nz/aplac/documents/web_docs/APLACPT003.pdf - Accreditation & Quality Assurance journal

  8. ISO/IEC 17025 and PROFICIENCY TESTING

  9. ISO/IEC 17025 and PROFICIENCY TESTING • PT-testing is not about ‘passing’ or ‘failing’ a test. • PT-testing is about taking part and learning from the results • One bad result does not make a laboratory bad. The challenge is to prevent that it repeats • One good result does not make a laboratory supreme. The challenge is to repeat this performance • Goal: consistent satisfactory performance

  10. ISO/IEC 17025 and PROFICIENCY TESTING Auditors : KEEPIN MIND • PT participation done under “daily conditions” ? • Performance over sevral years with same PT organizer • PT operated in accordance to ISO/IEC Guide 43-1 • The scheme documentation and statistical protocol available ?

  11. ISO/IEC 17025 and PROFICIENCY TESTING Does any PT-scheme exist for the tests and samples normally analyzed by the laboratory ?What IF NOT ? • - Analysis of blind samples • - Analysis of spiked samples • - Analysis of multiple sub-samples • - Exchange of samples with other laboratories • - Use of more than one analytical method

  12. WRONG OBJECTIVES QC/QA and method validation ?…. PT IS OFTEN A WRONG CHOICEAuditor: Check the method validation !! For assessment of sources of error ?….PT CAN BE A WRONG CHOICEAuditor: Check how and who did the PT measurement Because their clients say so ?….BUT WILL THEIR CLIENTS JUDGE ON BASIS OF 1 TEST ONLY ? Auditor: Check which client asked for this To get an impartial and independent view of the laboratory’s performance ?…. PT DOES NOT INDICATE THE REAL (DAY-BY-DAY) PERFORMANCEAuditor: Explain the PT report Because they have no suitable calibrants ?…. POSSIBLY, BUT CREATING A NEW RM BY PT IS A TIME-CONSUMING PROCESS…Auditor: What did they do so far ?

  13. Proficiency testing : insight in the REAL performance ? Ref. R.Forman, Pittcon 2002 As PT test A B C D E Aroclor 1242 120 % 128 % 102 % 102 % 111 %benzo(k)fluoranthene 210 % 0 0 0 210 %chrysene 139 % 94 % 77 % 72 % 158 %arsenic 138 % 106 % 91 % 103 % 112 %cadmium 138 % 100 % 89 % 104 % 102 %lead 132 % 91 % 99 % 112 % 115 % As double-blinded study A B C D E Aroclor 1242 0 62 % 0 0 143 %benzo(k)fluoranthene 0 55 % 119 % 0 55 %chrysene 0 59 % 73 % 0 49 %arsenic 84 % 55 % 99 % 57 % 61 %cadmium 10 % 12 % 98 % 20 % 25 %lead 27 % 15 % 88 % 16 % 34 %

  14. ISO/IEC 17025 and PROFICIENCY TESTING • RELEVANT DOCUMENTATION • Planned action • List of PT’s in which is participated • SOP’s for selection participation,administration, evaluation and follow-up

  15. ISO/IEC 17025 and PROFICIENCY TESTING VALIDITY OF RESULTS OF PT The period of validity of the result obtained by a laboratory in a single round of a PT scheme is limited to the time that the laboratory performed the test !! If a laboratory achieves a satisfactory result in a single round, the result shall not be used to support a claim that the laboratory obtained reliable data on any other occasion !!!

  16. ISO/IEC 17025 and PROFICIENCY TESTING

  17. ISO/IEC 17025 and PROFICIENCY TESTING

  18. ISO/IEC 17025 and PROFICIENCY TESTING

  19. ISO/IEC 17025 and PROFICIENCY TESTING

  20. ISO/IEC 17025 and PROFICIENCY TESTING CHECKLIST FOR AUDITORS • Is the PT-scheme in which they participated relevant in view of the scope of accreditation ? • Check: • The normal IQC and results thereof: frequency, control charts, criteria, evaluation, remedial/corrective actions • Type of material in daily measurement vs. IQC material vs. PT material • Frequency of measuring routine samples of the type supplied as PT • Measurand • Measurand value in comparison with typical ‘daily’ range • Deadline for reporting (turn-around time in comparison with routine work) • Who analyzed it (senior staff member vs. technician ?) • (Non) availability of IQC material -what else for routine control ? • (Non) availability of suitable calibrants - how method validation ? • Number of participants • Expected number of different techniques and/or number of techniques, similar as applied in the laboratory • Sample dimensions comparable to daily work conditions ? • How may the majority have measured it: best measurement capability or routine? • Frequency of this type of PT testing • Results of previous PT’s on same material

  21. ISO/IEC 17025 and PROFICIENCY TESTING DON’T FORGET: • If the PT score is poor (|z| > 2 or |z|>3) : • Assess how the target uncertainty was determined; compare this with e.g. the associated value predicted by the Horwitz equation Sometime target uncertainties are unrealistic small, set by metrologists under best measurement conditions, not taking instead of “routine” conditions • Check your own measurement uncertainty

  22. ISO/IEC 17025 and PROFICIENCY TESTING STUDY THE OVERALL PERFORMANCE OF ALL PARTICIPANTS: Unsatisfactory where the majority is satisfactory ?Unsatisfactory where a significant number performed poorly ? Both : indications of problems regarding methodology… ? then re-validate criteria for satisfactory performance incorrectly set ? then check performance of analysts….

  23. ISO/IEC 17025 and PROFICIENCY TESTING IS IT JUSTIFIABLE TO EXCLUDE OUTLIERS ? ISO-34 Clause 5.15.1 “…The reference material producer should never rely entirely on a statistical analysis of the characterization data when assessing the property values of interest. Outliers should not be excluded on purely statistical evidence until they have been thoroughly investigatedand, where possible, the reasons for the discrepancies identified. Alternatively, the use of robust statistics may be appropriate in some cases….”

  24. ISO/IEC 17025 and PROFICIENCY TESTING DO WE NEED STATISTICS IN PT ? Yes: - For “objective” demonstration of performance - For “objective” comparison of laboratories/ techniques/methods - If PT material has to be upgraded to RM No: - If participants use results for finding deficiencies - If fast turn-around times are necessary

  25. ISO/IEC 17025 and PROFICIENCY TESTING Bad news:Many things in nature are not “normally” distributed.Good news:Much of what is not “normally” distributed in biology would be “normally” distributed if you took the logarithm of each data item. Bad news:Much of what is in books about statistics has to do with “normally” distributed data. Good news:Robust statistics provide useful information even if applied to not “normally” distributed data.

  26. ISO/IEC 17025 and PROFICIENCY TESTING WHY DO ACCREDITED LABS PERFORM NOT BETTER THAN NON-ACCREDITED LABS? BECAUSE THESE COMPARISONS BETWEEN LAB-PERFORMANCES ARE BIASED !!! 1. Quality conscious laboratories join PT as part of their quality improvement strategy; other, less conscious laboratories are less likely to participate. 2. Non-accredited laboratories that perform badly may more likely to drop participation in PT than accredited labs, who need it to maintain their accreditation. So: there is an inclusion of only “good” non-accredited laboratories with both “good” and “bad” accredited laboratories !! 3. Accredited laboratories may even have no motivation to perform ‘super’ good;results are fit for the (clients) purpose4. Accredited laboratories may have much more pressure on routine performance (large throughput) than non-accredited laboratories

  27. Thank you for listening

More Related