Practical Solutions to Traceability and Uncertainty in Accreditation

1. Practical Solutions to Traceability and Uncertainty in Accreditation Presented to CITAC-NCSLI Joint Workshop “Traceability and Uncertainty: Key Technical Issues and Laboratory Accreditation” PITTCON 2002, New Orleans Sunday, 17 March 2002 Warren Merkel, A2LA Technical Manager

2. Presentation Overview • Relevant ISO/IEC 17025 Requirements • Accreditation Body Policies • Practical Approaches for Laboratories • Future Developments • Questions American Association for Laboratory Accreditation

3. Measurement Traceability - 17025 Requirements • Calibrations & reference materials traceable to SI where possible • Where not possible, traceable to certified reference materials, agreed methods and/or consensus standards • …Unless it has been established that the associated contribution from the calibration contributes little to the total uncertainty of the test result American Association for Laboratory Accreditation

4. Traceability- Definition • Relation to stated references through an unbroken chain of comparisons… • Traceability to some stated reference available with most RMs • Normally not SI unit for amount of substance, but often other SI units • All having stated uncertainties • Little evidence of verification of uncertainty claims American Association for Laboratory Accreditation

5. Traceability in Practice • In many cases, traceability to the test method is all that is required by clients • Uncertainty statements on RMs are important, but usually not a significant contributor to uncertainty of test • Performance in proficiency tests can serve as an indicator of traceability problems American Association for Laboratory Accreditation

6. Measurement Uncertainty - 17025 Requirements • 5.4.6.2 - Testing laboratories shall have and apply procedures for estimating uncertainty of measurement • Nature of the test method may preclude rigorous, metrologically and statistically valid, calculation of uncertainty of measurement • Laboratory shall at least attempt to identify all uncertainty components, make a reasonable estimation, and ensure that the form of reporting of result does not give a wrong impression of uncertainty American Association for Laboratory Accreditation

7. Measurement Uncertainty - 17025 “Loopholes” • Note 1 - Degree of rigor depends on: • Requirements of the method • Requirements of the client • Existence of narrow limits on specification conformance • Note 2 - In cases where a well-recognized test method specifies limits to the values of the major sources of uncertainty and specifies the form of presentation of calculated results, the laboratory is considered to have satisfied this clause by following the test method and reporting instructions American Association for Laboratory Accreditation

8. Measurement Uncertainty - 17025 Reporting Requirements • 5.10.3.1 c) Information on uncertainty is needed in test reports when it is relevant to the validity or application of the test results, when a client’s instruction so requires, or when the uncertainty affects compliance to a specification limit • 5.10.1 - In the case of a written agreement with the client, results may be reported in a simplified way American Association for Laboratory Accreditation

9. A2LA Interim Policy on Measurement Uncertainty for Testing Laboratories • Five categories of test methods • Intended to facilitate transition- pragmatic approach, not ideal in all cases • Results from review at annual meeting: • Some modification of language • Publish list of example methods for each category by field of testing • Publish guidance on determining uncertainty in testing based on ISO 5725 American Association for Laboratory Accreditation

10. A2LA Interim Policy on Measurement Uncertainty for Testing Laboratories • Five categories of test methods: I. Qualitative No uncertainty calculations required Examples: Ignitability; Microbiological screening II. Well-recognized methods that specify limits to uncertainty contributions (Note 2) No further uncertainty calculations required Examples: Flash point; Hardness Problems: Modification of method What if a client wants uncertainty? American Association for Laboratory Accreditation

11. A2LA Interim Policy on Measurement Uncertainty for Testing Laboratories • Five categories of test methods, continued: III. Published methods that do not specify limits to uncertainty sources and/or reporting format Uncertainty estimated using standard deviation of laboratory control samples Examples: Alloy analysis by OES; VOA Problems: Normal process for analyzing control samples may lead to an underestimate of uncertainty Quality of control sample American Association for Laboratory Accreditation

12. A2LA Interim Policy on Measurement Uncertainty for Testing Laboratories • Five categories of test methods, continued: IV. Methods requiring identification of major uncertainty components and reasonable estimate of uncertainty Examples: PBMS; One-off tests V. Methods requiringfull uncertainty analysis consistent with ISO Guide to the Expression of Uncertainty in Measurement Example: Reference material value assignment American Association for Laboratory Accreditation

13. A2LA Interim Policy - Chemical Laboratories • Majority of methods classified as Category II, III, IV • Classification can vary by laboratory and use • Use of RMs critical to demonstrating process control and evaluating bias • More practical guidance required American Association for Laboratory Accreditation

14. 17025 Requirements: Quality Control and Proficiency Testing • Laboratory shall have procedures for monitoring validity of tests, including: • regular use of CRMs or internal QC using secondary reference materials • participation in interlaboratory comparison or proficiency testing programs • Record data so trends are detectable • Laboratories in most cases already have sufficient data for estimating uncertainty American Association for Laboratory Accreditation

15. Practical Approach - Category II • Laboratories performing tests in Category II can utilize precision data published with method as uncertainty estimate if: • Laboratory has data demonstrating that its repeatability is comparable to the method data • Material used for precision estimate is similar to materials tested by lab • Method not modified • Basis for estimate clearly stated American Association for Laboratory Accreditation

16. Practical Approach - Category III • Intermediate measures of precision provide adequate estimate of uncertainty, if: • Measurement method standardized • Measurement process is in control • Control sample well characterized (preferably CRM) • Process for collecting data designed to vary all significant uncertainty components American Association for Laboratory Accreditation

17. Steps Laboratories Can Take • Identify major uncertainty components (App. D of EURACHEM/CITAC Guide) • Establish control charts (ISO 8258) • Attempt to design QC process to ensure representative variation of inputs • Fewer points  more emphasis on design • May need to record additional data related to precision as objective evidence American Association for Laboratory Accreditation

18. Steps Laboratories Can Take • When required to report uncertainty, clearly define the basis for the estimate • Clarify during contract review process • If possible, determine with client end use of data • Make use of PT study data • Compare internal precision data to spread of results of participants • If study based on reference value, compare lab result/uncertainty with reference American Association for Laboratory Accreditation

19. Future Developments • Consensus method development • More rigor in precision data • Additional detail regarding uncertainty sources • Accreditation of RM producers • Increased focus on PT based on reference values vs. consensus values • Increased awareness of traceability uncertainty issues in user community American Association for Laboratory Accreditation

20. Conclusions • Accreditation bodies and laboratories reconciling 17025 requirements that are ahead of the state of development in many industries • Pragmatic approach to requirements is necessary • Goal: provide data that is fit for purpose American Association for Laboratory Accreditation

21. Contact Information Warren MerkelA2LA5301 Buckeystown Pike Suite 350Frederick, MD, USA 21704 Direct line: 301 644 3204 Main: 301 644 3248 Fax: 301 662 2974 wmerkel@a2la.org www.a2la.org American Association for Laboratory Accreditation