How Proficiency Testing helps to reduce the impact of incorrect microbiology results on public health Heena Shah and Ni

1. How Proficiency Testing helps to reduce the impact of incorrect microbiology results on public health Heena Shah and Nita Patel26 September 2013

2. Contents • Public Health England • Proficiency Testing • Challenging organisms/samples • Improvement to schemes • On-going performance assessment • Summary Proficiency Testing

3. What is Public Health England (PHE) • Public Health England (PHE) is an executive agency of the United Kingdom’s Department of Health. • PHE is the expert national public health agency which fulfils the Secretary of State for Health’s statutory duty to protect health and address inequalities, and executes his power to promote the health and wellbeing of the nation. Proficiency Testing

4. Public Health England Public Health England (PHE) will work to help people live longer, healthier lives by supporting and enabling local government, the NHS and the public to protect and improve health and wellbeing and reduce inequalities. • PHE will bring about fundamental improvements in the public’s health and wellbeing, by focusing its energies on five key priorities: • Helping people to live longer by reducing preventable deaths from conditions such as heart disease, stroke, cancer and liver disease Proficiency Testing

5. Public Health England cont. • Increasing healthy life expectancy by tackling conditions which place a burden on many lives, such as anxiety, depression and back pain • Protecting the population from infectious diseases and environmental hazards, including emerging risks and the growing problem of antimicrobial resistance • Supporting families to give children the best start in life, through working with health visitors, Family Nurse Partnerships and the Troubled Families Programme • Helping employers to facilitate and encourage their staff to make healthy choices Proficiency Testing

6. Public Health England……. • ...recognises the role that food, water and clinical microbiology laboratories play in protecting people’s health so provides tools to support laboratories in assuring their results. • Tools provided are proficiency testing and reference materials. • Errors may be reduced if a robust quality system is in place that includes external assessments of laboratory processes and effective internal quality control ????? Proficiency Testing

7. External Quality Assurance Department (e-QAD) • Specialise in proficiency testing for microbiology • Two specialist areas: • Food and water microbiology (organised by the Food and Environmental Proficiency testing Unit - FEPTU) • Clinical microbiology (UK NEQAS for Microbiology) • >50 staff • Over 45 international schemes available Proficiency Testing

8. Why PHE provides EQA/PT schemes? • PHE’s goal is to prevent and reduce the incidence and consequences of infection • Food, water, environmental and clinical microbiology laboratories play a vital role in protecting people’s health • by ensuring food is safe to consume • by ensuring recreational waters do not make people ill • by ensuring that water is safe to use or drink • by identifying the causes of peoples illnesses • All PHE EQA/PT schemes aim to educate and emphasise that reporting incorrect results can have severe consequences Proficiency Testing

9. Why laboratories do PT • Demonstrate competence as part of accreditation requirement – ISO/IEC 17025:2005 - General requirements for the competence of testing and calibration laboratories • Helps to provide assurance of the results obtained provided they are treated and processed the same as other samples • Helps improve laboratory processes and understanding of regulation/legislation • To remain up to date with new and emerging organisms - educational • To challenge processes/media/training with difficult or atypical organisms • Inter-laboratory comparison of performance • To support work tendered for as an accredited laboratory Proficiency Testing

10. Importance to testing laboratories • Unbiased assessment of performance (Z scores and PHE scores) • Laboratory to have confidence in results obtained • Reassures clients on the laboratory’s testing ability • Laboratories that identify problems understand the direct impact on public health • Confidentiality of performance Proficiency Testing

11. Food and Environmental Proficiency Testing Unit (FEPTU) Proficiency Testing

12. Food and Environmental Proficiency Testing Unit (FEPTU) • Team of 14 staff - PT experts, microbiologists, data analysts, administrators and support staff • Supported by other PHE staff including: • Statistician • Quality assurance managers • Reference laboratories • Business manager • IT support • Epidemiologists and food law experts • Food and water microbiology consultants • International Office, Press and Publications Proficiency Testing

13. External quality assessment (EQA) / Proficiency Testing (PT) • Samples of known but undisclosed content are introduced into a laboratory’s routine testing procedure LENTICULE® discs Freeze-dried Proficiency Testing

14. Proficiency Testing ORGANISING LABORATORY PARTICIPANTS Prepare samples Examine samples Analyse results Report results Evaluate Prepare report Proficiency Testing

15. Impact of incorrect results – food microbiology • FALSE NEGATIVE • Reporting incorrectly that a sample does not contain a pathogen • Unsafe product released for sale • People become ill/die • Negative publicity • Loss of reputation • Financial cost • FALSE POSITIVE • Reporting incorrectly that a sample does contain a pathogen • Product recall • Incorrect product withdrawal • Ban on export • Loss of contracts • Financial cost Presentation title - edit in Header and Footer

16. External quality assessment • EQA/PT samples should be treated the same as routine samples • must be examined by the same staff who normally examine the routine samples • normal routine methods must be used • checks on reporting should be the same as for routine samples • Evaluation of results • was the result correct? • if not what was the cause? • what did the laboratory do to improve? Proficiency Testing

17. Why a laboratory may report an incorrect results • Sample handling errors • Inadequate method • Lack of understanding of guidelines and legislation • Inadequate staff training • Equipment and culture media failures • Calculation errors • Reporting errors • These mistakes can be reduced if a robust quality system is in place Proficiency Testing

18. Schemes Water PT • Legionella Isolation Scheme • Drinking Water Scheme • Recreational and Surface Water Scheme • Endoscope Rinse Water Scheme • Bottle and Mineral Water Scheme • Dialysis Water Scheme • Hospital Tap Water Scheme • CompetencyCheck samples • Food PT • Standard Scheme • Public Health Scheme • European Food Microbiology Legislation Scheme • Non-Pathogen Scheme • Shellfish Scheme • Pathogenic Vibrio Scheme • Staphylococcus aureus Enterotoxin Scheme

19. Challenging organisms/samples Proficiency Testing

20. Standard Scheme • Laboratories that routinely test for a range of food-borne pathogens and indicator organisms • Participants are often private laboratories that test foods for clients in the food industry who may submit products routinely for microbial assessment, end product testing and customer complaints. • Pathogens Enumerations • Salmonellaspp. Presumptive Bacillus cereus • Listeria spp. Campylobacterspp. • Escherichia coli O157 (non-toxigenic strains) Clostridium perfringens • Campylobacter spp. Coagulase-positive staphylococci • Listeria monocytogenes • Listeriaspp. • Aerobic colony count • Escherichia coli • Enterobacteriaceae • Coliforms Proficiency Testing

21. Sample S0521 – Standard Scheme • Distribution February 2013 • Wide range of enumeration results reported for coliform 2.1x104- 3.0x105cfu g-1(median 9.5x104), 21 outlying results • Sample content: • Staphylococcus aureus • Listeria monocytogenes • Bacillus pumilus • Cronobacter sakazakii • Pantoea agglomerans • Escherichia coli Proficiency Testing

22. Sample S0521 – Standard Scheme • Cronobacter sakazakii and Pantoea agglomerans both produce 1mm purplish red colonies on violet red bile lactose agar. • E. coli Proficiency Testing

23. Sample S0521 – Standard Scheme • ISO 4832:2006 Microbiology of food and animal feeding stuffs - Horizontal method for the enumeration of coliforms - Colony-count technique recommends to count the purplish red colonies with a diameter of at least 0.5mm. These are considered as typical colonies of coliforms and do not require further confirmation. The ISO document further states to count and confirm atypical colonies (e.g. of smaller size). • If participants did a confirmation test: • C. sakazakii would not have produced gas in brilliant green bile broth (BGBB) → not a coliform • P. agglomerans produces gas → coliform • Variation in the result may also be due to the different methods used • Raising awareness of atypical organism and subsequent impact on reported results Proficiency Testing

24. Sample S0530 – Standard Scheme • Distribution June 2013 • This sample contained: • Salmonella Liverpool • Enterococcus faecalis • Micrococcus spp. • Providencia alcalifaciens Proficiency Testing

25. Sample S0530 – Standard Scheme • Enterobacteriaceae parameter • A large number of participants failed to achieve the expected results. • Standard deviation was 0.7 log10 cfu g-1 and there was a wide range of results reported (10 – 1.3x103) • Homogeneity results performed by FEPTU prior to distribution gave counts of 1x103 cfu g-1, whilst tests performed during the distribution period gave results of 40 - 150 cfu g-1. • This parameter was not scored. Proficiency Testing

26. Sample S0530 – Standard Scheme • Findings: • 1. There is evidence to suggest from in-house quality control results on the samples and the participants’ reported results that this sample is unstable • 2. The Providencia alcalifaciens in this sample produces purple/pink 0.5 – 2mm colonies on Violet Red Bile Glucose agar (VRBG) with no halo, this strain is a weak/late glucose fermenter (glucose stab incubated 18 hours at 37oC) Providencia alcalifaciens Proficiency Testing

27. Sample S0530 – Standard Scheme • Participants doing a confirmatory test may have reported the P. alcalifaciens as negative. This may have also accounted for the wide variations of results observed with this sample. • Participants are reminded that ISO 21528-2:2004 Microbiology of food and animal feeding stuffs -- Horizontal methods for the detection and enumeration of Enterobacteriaceae - Part 2: Colony-count method states that if a yellow colour develops throughout the contents of the tube, regard this reaction as positive. • Atypical organism, raising awareness of ISO method and impact on reported results Proficiency Testing

28. Sample S0530 – Standard Scheme • However the draft ISO 21528 – 2 Providencia alcalifaciens Proficiency Testing

29. European Food Microbiology Legislation Scheme • Laboratories that examine foods for compliance with Regulation (EC) 2073/2005 and subsequent amendments • The scheme assesses participants' ability to test and interpret laboratory results in accordance with EU food safety and process hygiene criteria • PathogensEnumerations • Cronobacter sakazakii Presumptive Bacillus cereus • Salmonellaspp. Coagulase-positive staphylococci • Listeria monocytogenesListeria monocytogenes • Aerobic colony count • Escherichia coli • Escherichia coli MPN • Enterobacteriaceae • Staph enterotoxin Proficiency Testing

30. Sample EFL074 – European Food Microbiology Legislation Scheme • Distribution May 2013 • Sample details: A sample of uncooked turkey breast taken from the market within the shelf-life of the product. • Request: Test to determine whether this sample complies with the food safety criteria stipulated in Commission Regulation (EC) No. 2073/2005 and subsequent amendments. • Sample content: • Salmonella Enteritidis • Aeromonas hydrophila • Escherichia coli Proficiency Testing

31. Sample EFL074 – European Food Microbiology Legislation Scheme • This sample was designed to test participant’s understanding of Commission Regulation (EU) No 1086/2011 food category 1.28. The testing requirements are to determine the presence or absence of Salmonella Typhimurium and Salmonella Enteritidis from a 25gram sample. Proficiency Testing

32. Sample EFL074 – European Food Microbiology Legislation Scheme • 24 laboratories examined this sample for Salmonella spp. • All reported the presence of this organism, with five reporting a S. Enteritidis. • 19 laboratories reporting a result as Salmonella spp. • - only, six indicated that the strain would be sent to a reference laboratory for further confirmation on the species identification and serotype to determine compliance. • One participant additionally examined this sample for Listeria monocytogenes which is not required to determine compliance with the food safety criteria. Proficiency Testing

33. Sample EFL074 – European Food Microbiology Legislation Scheme • Participants are reminded that to ensure compliance with this part of the regulation, they must examine the sample as required for the test criteria. • Sample was sent out for educational purposes to raise awareness of the amendment to regulation (EC) 2073/2005, sample was not scored. • Raising awareness of updates in legislation and testing requirements Proficiency Testing

34. Non-Pathogen Scheme • Laboratories on production sites that do not want to introduce pathogens • Aerobic colony count • Coliforms • Escherichia coli • Enterobacteriaceae • Enterococci • Lactic acid bacteria • Presumptive Pseudomonas spp. • Yeasts • Moulds Proficiency Testing

35. Sample NP0123 - Non-Pathogen Scheme • Distribution March 2012 • Sample content: • Pseudomonas fluorescens • Candida fomata • Penicillium spp. • Pantoea agglomerans • Escherichia coli • Leuconostoc spp. Proficiency Testing

36. Sample NP0123 - Non-Pathogen Scheme • NP0123 – E. coli enumeration • 27/30 laboratories reported a negative result • The E. coli strain included in this sample was β-glucuronidase negative. • 3% of E. coli strains do not produce glucuronidase • Raising awareness of atypical organisms that could occur in food samples, and the consequential impact of the method used. Proficiency Testing

37. Improvements to schemes Proficiency Testing

38. Method based presentation of results • Introduced method based presentation of results in the Drinking Water Scheme from April 2013. • For each parameter examined the graphs generated help participants to assure the methods they use and allow comparability of the results returned by other participants for methods used or available. Proficiency Testing

39. Method based presentation of results Proficiency Testing

40. Method based presentation of results • For 2013 • Extend providing this data in other schemes: • Non-Pathogen Scheme • Standard Scheme Proficiency Testing

41. Pathogenic Vibrio Scheme • Scheme designed for participants that examine food samples for Vibrio spp. • Parameters: • Vibrio parahaemolyticus • Vibrio cholerae (non-O1) • Addition of Vibrio vulnificus to the testing parameters as this organisms is increasing becoming a problem across Europe. V. vulnificus can cause disease in those who eat contaminated seafood especially oysters. Proficiency Testing

42. On-going performance assessment • Allocation of scores is provided as a management tool to help assess performance. It cannot replace results and methods and should never be amended just to achieve better scores • Is designed to alert participants with on-going problems • Only useful if a laboratory takes more than one distribution a year • Allows laboratories to Identify errors in the system • It is an effective tool for on-going monitoring of quality control processes • Outlying results are allocated a low score • However on single occasion participants must not be alarmed but they should access the reasons why? Proficiency Testing

43. On-going performance assessment Proficiency Testing

44. On-going performance assessment Proficiency Testing

45. Summarise benefits of PHE PT • We offer unique schemes • We use wild type strains to prepare samples not NCTC or ATCC • Repeat samples provided free of charge with the next distribution • On-going performance assessment is provided in reports • Advice from expert microbiologists and consultants available • Method based presentation of results • Improve understanding of guidelines/legislations • Our samples are challenging and can be educational when atypical strains are used Proficiency Testing

46. Summary • PHE’s role is to help protect the health of everyone in the UK • Food and water examination laboratories play a vital role in protecting people’s health by ensuring food is safe to eat and water is safe to drink and use • PHE provides EQA / PT to help laboratories assure the results reported • Schemes have an educational aspect and are continually being improved to meet participants requirements • Laboratories should take part in more than one PT distribution a year to provide assurance on the results reported throughout the year (and to cover the range of organisms/levels) • EQA / PT schemes and internal quality control processes using reliable reference materials are essential elements of a laboratory quality system Proficiency Testing

47. Acknowledgements • Richard A’Court • Nick Andrews • Melody Greenwood • Dr Jim McLauchlin • eQAD clinical team • Morolake Adedeji • Richard Borrill • Kermin Daruwalla • Stephanie Foster • Carmen Gomes • Wilson Gomes • Zak Prior • Manchari Rajkumar • Thamayanthy Ramesh • Vince Smith • Aneta Stranc • Anitha Tallam 47

48. Thank you for listening Proficiency Testing