CTL WORKSHOP 2008

1. CTL WORKSHOP 2008 Session 3 Concepts What else do you do with your proficiency testing data? Mini-audits prior to performing PTPs Mini-audits and CSP

2. Group Task - (from earlier today) • Describe to one another how proficiency tests (internal and external) are used within your respective organisations. • Prepare a list of these uses. • Discuss what kinds of things you would further like to gain from proficiency testing programs, and list these • How could use of PT be facilitated by • PT provider • Management within your organisation

4. Mini-audits – what are they? • Equipment checks to specific areas sourced both from the relevant standards and from CTL decision sheets (& other documents) • Performed prior to performing the PTPs. • Participants have the opportunity to correct deficiencies prior to performing the tests • Reduce the chances of gaining a non-compliance after the PTP

5. Mini audits during 2007 • Glow wire test/ needle flame test • Ball pressure test • These topics have operated several times before and knowledge exists about the reasons for outlying results

6. The mini-audit format was optional: • Participants could send these results first and test after they obtained out evaluation report or, • They could complete the tests and the audits together. • The programs were very efficient (turn around time for reporting was maximum 30 days) • Audit results were provided within a few days of submitting results and were known before issue of the main report.

7. Participants • These programs were originally intended for manufacturer’s labs • CTL decided in 2007 to include these programs on the “official” list • The participants that had no equipment non-conformities or that corrected their equipment issues prior to testing had a smaller proportion of outliers than those performing the tests and submitting the mini-audit results together. (50% better results)

8. Why have an equipment audit? • Equipment compliance is as much part of compliance with the standard as performing the test (maybe more so) • It is clear that the testing staff in many laboratories do not have sufficient knowledge of equipment requirements • Equipment audits have been “standard” in most PTPs operated for CTL. • Performing the equipment audit prior to starting the test is logical

9. 2008 and 2009 • The format of certain programs will include equipment mini-audits for CBTLs as a trial. • Participants will be encouraged to do the equipment audit and get feedback (and act on it if necessary) prior to performing the test.

10. Lead times for program starts may be increased to allow this to happen. • However, the final reports will be issued MUCH earlier • Follow up activities will still be required for equipment non-compliances, particularly if not addressed by the time the final report is issued • CTL can feedback on this next year and we will jointly decide how to proceed after this.

11. Accreditation Requirements • ISO IEC 17025 requires proficiency testing to be performed. • The same clause in 17025 describes other techniques for assuring quality of results. • Duplicate testing by the same staff • Possible using PT samples when sufficient samples are provided or when non-destructive tests are done – but PTPs are rare compared with the volume of normal tests performed. • Testing the same sample by multiple staff • PTP samples and other samples • Testing standard samples (with known results) in parallel. • Ex-PTP samples, purchased standard materials (rare in electrical tests), some retained samples

12. Why do these requirements exist? • It is normal quality assurance • A laboratory should know that: • Any staff member can reproduce his/her own result when the same sample is tested on another occasion • Any staff member will gain an equivalent result to all other staff members in the same laboratory • The laboratory as a whole should obtain equivalent results to other laboratories. (This is PTP)

13. Organisation and Commitment • Some people in this field argue that time is money, it takes time to perform testing, and every minute spent testing quality assurance samples means lost business.

14. People in some other fields argue that they cannot afford NOT to do quality assurance tests as they need proof of testing integrity and consistency of results in order to be confident when they sign their name on a test certificate. • However, many people in all fields say they have difficulty planning, organising , analysing and reporting their full quality activities in a manner that management can quickly grasp what is presented (gobbilty gook)

15. Many organisations use their PTP programs as • Opportunities to meet some of the additional 17025 requirements • PT samples not subjected to destructive tests can be tested again and again, by different staff and at different times.. • The expected results and allowable limits are known from the final report • Participants should retain samples used for non-destructive testing for these purposes

16. What other uses for retained samples? • Staff training • Equipment function checks • Calibration checks

17. What about destructive tests? • ?? • Retained samples can be tested again, but these are limited in number and not offered regularly • IFM operates check sample programs for other disciplines of testing

18. What is a check sample program? • Series of shorter proficiency tests designed to allow multiple topics to be covered for multiple operators within the same laboratory • Planned and organised proficiency tests that can cover additional requirements from 17025

19. Drawing on others’ experience • Laboratory A has 3 staff trained to carry out various tests in a particular area • When a PT program is offered in this topic, all 3 staff have input (observe), but only one staff can actually carry out the test. • It may be 3 years until the topic is operated again, but another staff gets to perform the test that time. • Realistically, even with staff rotation on PTPs, some staff may never really be tested, and it could take 10 years before the same staff is tested again.

20. Alternatives: • Lab can order more than one sample when the PTP is operated • Staff still only test themselves every 3 years or so, and no catering for new staff in the meanwhile • Choose a different format of program

21. Typical IFM check sample program • Operates for 6 or 12 months • Involves 3-8 tests • All tests are performed on a matrix every 1-2 months (6-12 samples in all) • Designed to make quality assurance testing routine – samples would be treated as all others are (true reflection of the capabilities of the lab)

22. Duplicate tests and repeat tests are built into the program • Multiple testing staff can be catered for in the same program (can either be “rostered” to perform tests on various rounds, or can share samples in a round)

23. Testing can be fitted to normal day to day tests, as are simpler in format and paperwork • Results are entered on-line by the due date. Each individual operator has their own data entry page • A simple and rapid turn-around report is generated for each round, and before the next round starts

24. Labs can implement and gauge effectiveness of any corrective actions more rapidly than with conventional program design because they have a repeat sample rapidly following • At the end of the cycle, each laboratory obtains a summary report for the cycle, including individual breakdown of each operator

25. This style of program promotes consistency of performance because: • QA testing becomes routine • Quality issues have a greater focus in the lab (because testing staff are involved more often) • Laboratories gain capability to trend their performance (another 17025 requirement!) • Individual and group staff training needs are easier to identify

26. For electrical test laboratories • Required to pass equipment audit prior to enrolling • Is more time efficient for labs than the regular style of program • IFM will be trialing glow wire test, needle flame test and ball pressure test as a check sample program starting in September/October 2008

27. Laboratories can enroll in this program as an alternative to 08e29 and 09e34. (Total cost will be approximately 25% less). • The technical requirements of the program will be exactly the same. • It is hoped that the check sample program will seem less onerous, as the testing requirements will be spread over 6 months • Organisations wishing to volunteer for this trial should contact IFM ASAP for more details

28. Summarising: • 08e29 and 09e34 will be offered in 2 formats: • traditional program with equipment mini-audit • Combined check sample program (equipment audit still required prior to commencing) • Laboratories enrolling in both programs can opt for either format • Feedback on the experience will be presented in 2009 CTL and workshop to decide how to proceed • Volunteers are required!

29. Open discussion on any matter