Observations from 20 years of PT in South Africa

1. Observations from 20 years of PT in South Africa Dr Jim McCulloch Thistle QA Johannesburg

2. Who am I? • Worked in labs from 1965 • Experienced and qualified in Microbiology, Chemistry, Statistics and Clinical Chemistry • Msc Medicine, PhD in Clinical Chemistry • Setup Thistle QA in 1990

3. What is Thistle QA all about? • Started with 1 EQA (PT) in 1990 and with 1 employee, me! • Now... • 23 PTs, 12 staff members • 2500 labs directly enrolled • Data base has results from 60 countries • Data base number over 6000

4. Why would you want PT anyway? PICK THE CORRECT ANSWER • ISO says we need it • SANAS will look for it • It helps achieve a quality result

5. Aims of PT • To determine the performance of individual labs for specific tests • To monitor the continuing performance of labs • An essential component of a good quality system

6. PT Design • Reference values or consensus • Defined methods? • Outlier detection and removal • Statistics to use, robust or not? • Size of data base • Report issues • Customer complaints

7. Outliers • Wild results, bummers, etc • Chauvenet’s Criterion • Grubb’s Test • Robust statistics using ISO 13528: 2005, e.g. iteration

8. Complaint No 1 from 1990 • (We) will not be participating in cycle three of what you euphemistically call your quality control scheme. • Quaint statistical manipulation and unacceptable transcription errors render your scheme more than a little suspect. • Please note that we do not require reports. I would sooner the paper be saved for a more useful function than cluttering up my waste paper basket.

9. Bolt on extras • Support • Teaching material • Seminars • Extra samples • Custom designed reports

10. And reports are stats • They are mere sets of figures, albeit with useful information • They give you warnings, not facts • You can be “out of control” and not have a non-conformance • Things are not always what they seem

11. Statistical cliché… • Myth: Mark Twain said: Lies, damned lies and statistics • Truth: Statistics don’t lie – it’s people who lie!

12. An example of wrong conclusion... Thistle QA

13. Don’t accept the surface

14. Get behind the stats Thistle QA

15. Why can you be okay and yet outside 2 SDs (z score >2)?

16. A control chart

17. What target value to use? • By calculation from formulation • CRM • RM • Consensus from reference labs • Consensus from participating labs

18. The current reports Thistle QA

19. A Review of Chemistry PT Analytes used • Sodium, measure of osmotic equilibrium • Glucose, likelihood of diabetes • Bilirubin, an assessment of liver function

20. Review of Chemistry PT Fitness for purpose – Acceptable Range • Sodium 3% • Glucose 7.4% • Bilirubin 20.0%

21. Review of Chemistry PT Methodology • Each lab’s results for an entire 6-month cycle • Reduced to one single figure, CV, basically, SD as a percentage of the mean / average result • Reviewed for changes per lab, and overall number of labs within the fitness-for-purpose percentage

22. Review of Chemistry PT Findings • Sodium 99.9% within f-f-p range • Glucose 99.3% within f-f-p • Bilirubin 99.1% within f-f-p General • IF first cycle out, next one always in f-f-p • UNLESS instrument changes

23. Review of Immunoassay PT • TSH, thyroid function test, assessment of thyroid stimulation • T4, thyroid function output test • PSA, prostate specific antigen

24. Review of IA (hormone testing) PT FFP • TSH, 20% • T4, 15% • PSA, 25% Methodology • As for Chemistry

25. Review of IA PT • TSH, 94.9% within f-f-p • T4, 89.9% within f-f-p • PSA, 95% within f-f-p

26. We all see things differently Thistle QA

27. The real thing

28. Conclusions • PT an essential tool • Needs education, interpretation • Can assist in complying with quality standard • Detects changes, e.g. Instrument • Cannot lead to CQI, rather compliance with f-f-p