1 / 37

User verification of performance

User verification of performance. Zahra Khatami /Robert Hill BHR University Hospitals/Sheffield Teaching Hospitals Focus 2013. Aim and objective. Provide an overview of measurement uncertainty Offer practically achievable procedures

signa
Download Presentation

User verification of performance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. User verification of performance Zahra Khatami/Robert Hill BHR University Hospitals/Sheffield Teaching Hospitals Focus 2013

  2. Aim and objective Provide an overview of measurement uncertainty Offer practically achievable procedures Recommend the minimum testing requirements also ensuring reliability Work through pre-programmed spreadsheets

  3. Communication language “International Vocabulary of Metrology” (VIM) JCGM 2008 www.bipm.org/utils/en/pdf/JCGM (BIPM, IEC, IFCC, IUPAC, IUPAP, ISO, OIML, ILAC)

  4. VIM definition Verification: Provision of objective evidence that a given item fulfils specified requirement Example: Confirmation that performance properties or legal requirements of a measuring system are achieved

  5. Common quandary Which performance criteria? What type of material? How many samples/measurements? How many days? What statistical tools?

  6. Performance properties In the context of clinical chemistry this equates to the uncertaintyassociated with the measurement

  7. VIM definition: Measurement Uncertainty Non-negative parameter characterizing the dispersion (imprecision) of the quantity values being attributed to a measurand (an interval of values within which the true value lies with a stated probability)

  8. VIM definition: Trueness Closeness of agreement between the averageof an infinite number of replicate measured quantity values and a reference quantity value (in absence of reference method the difference between specified methods)

  9. Which performance criteria? Precision (Imprecision) Trueness (Bias)

  10. Estimating the measurement uncertainty Guide to the expression of uncertainty measurement (GUM) Bottom–up approach which derives the uncertainty of a measurement result by combining the uncertainties related to the uncertainty sources of the measurement process

  11. Estimating the measurement uncertainty Guide to the expression of uncertainty measurement (GUM) Alternatively Top-down approach which uses the reproducibility as uncertainty estimate

  12. Reference publications EN/ISO 15189 (Requirements for quality & competence specific to the quality management system requirements particular to medical Laboratories) CLSI (Clinical Laboratory Standards Institute) Former NCCLS IFCC ECCLS (European Council for Clinical Laboratory Standards) obsolete since mid 90’s

  13. Reference publications CLSI EP15-A2 (User verification of performance for Precision and Trueness 2005) CLSI EP9-A2 (Method Comparison and Bias Estimation Using Patient Samples 2002)

  14. Reference publications Thorough document Lengthy document Large number of tests Detailed statistical processing Lack of accessible pre-programmed spreadsheets

  15. Authors • Zahra Khatami • Robert Hill • Catherine Sturgeon • Edward Kearney • Peter Breadon • Anders Kallner

  16. Aim Provide practical recommendations Minimum requirements/ensuring reliability User friendly Full pre-programmed spreadsheets Free access for all Regular review

  17. Measurement verification in the clinical Laboratory: A guide to assessing analytical performance during the acceptance testing of methods (quantitative examination procedures) and /or analysers http://www.acb.org.uk/An%20Ver/Measurement1.asp

  18. Assumptions Adequate verification by manufacturer, accessible by the user (IVD directive) Established performance specifications (precision, bias, linearity, measurement interval, interference, etc)

  19. Imprecision • What type of material? • Patient Pool or IQC material • How many measurements? • Five replicates at two concentrations • How many days? • Five days

  20. The statistical test ANOVA Tests for a difference between the means of a number of groupsof observations It partitions the total variance into the components: variation between the within and between each series of measurements It offers an estimate of within and between series variation

  21. Refer to Calcium data word file

  22. Bias: Patient comparison • How many measurements? 20 patient samples • How many days? Paired samples should be measured within a close time interval

  23. The statistical tests • T-test • Ordinary linear regression • Deming regression • Difference plot

  24. Refer to amylase word file

  25. Bias: using reference material • What type of material? Reference material with assigned value (CRM) • How many measurements? Two concentrations, duplicate measurements • How many days? 3-5 occasions/days • What statistical tool? T-test

  26. Bias: using EQA material • What type of material? Material with peer group values from EQA organizers • How many measurements? 7-10 concentrations, duplicate measurements • How many days? N/A • The statistical tool? T-test

  27. Thank you

  28. Everything should be made as simple as possible, but no simpler

More Related