Australia: Managing Testing Laboratories in Support of a Standards and Labelling Programme

1. Australia: Managing Testing Laboratories in Support of aStandards and Labelling Programme Bangalore - October 2004 Prepared by: Robert Foster (EES)

2. Presentation Overview • Background to AU program • Role of test laboratories • Management structure in AU • Role of a laboratory accreditation body • Development of Standards – case study • Compliance program in AU • Building capacity • Maintaining standards

3. Labelling: circa 1990 Refrigerators & Freezers Clothes Washers Clothes Dryers Dishwashers Air Conditioners - Single Phase MEPS: 1999 Electric Water Heaters Three-Phase Electric Motors Three Phase A/Cs MEPS: 2003 Fluorescent Lamp Ballasts MEPS: 2004 Commercial Refrigeration Linear Fluorescent Lamps Distribution Transformers Single Phase A/Cs Background -What is regulated in AU

4. Background – Regulatory Requirements • To be legally sold, products subject to Labelling & MEPS are required by law to: • Comply with the relevant AS/NZS Standards • Be registered with a state regulator via a national on-line database system • Test reports from laboratories are required to be supplied either: • With the registration application (labelled) or • Supplied only upon request by the regulator (MEPS) • Test laboratories supplying reports for registration purposes require no special accreditation, but a regulator may reject an application on the basis of the laboratory used (with grounds)

5. Background - Types of Laboratories Types • Own “in-house” • Independent Accreditation • NATA (National Association of Testing Authorities) • Other Overseas • None For every registration: Details of test laboratories used to certify product for registration purposes are logged in the national appliance database

6. Role of Test Laboratories in AU To assist in developing new, or adapting existing overseas standards for use in Australia (Trials) • Government needs confidence that test methods are: • Relevant to local conditions • Repeatable and reproducible • Practical to undertake • Cost effective • Industry needs confidence that standards are achievable and provide a level playing field for all • Consumers need confidence that standards will protect their interests

7. Role of Test Laboratories in AU To assist government to determine appropriate minimum performance levels (testing of representative product samples) • Government needs to be able to make its case that proposed minimum performance levels are: • Achievable • Have a positive benefit to cost ratio • Are relevant to local conditions • Industry needs confidence that new performance standards are achievable. In particular they want to know what they will need to do to meet those requirements

8. Role of Test Laboratories in AU To provide the necessary capacity for industry to meet their legal obligations • Government & Industry need to be confident that adequate infrastructure is in place to meet Industries testing needs. (especially for smaller players without in-house test facilities)

9. Role of Test Laboratories in AU To facilitate government compliance programs • Government needs capacity to test compliance. • Industry demands that its investment in Energy Efficiency is protected via a robust compliance program • Consumers require confidence that labelling and MEPS claims are reliable

10. Role of Test Laboratories in AU To provide technical input to standards committees (via laboratory staff secondment) • Both government and industry rely upon the technical expertise of leading laboratory personnel when formulating or modifying standards

11. NAEEEC AS/NZS Standards Standards Committees Standards Australia Federal (AGO) State (Regulators) Output Other (Govt) Policy Direction Program Management Technical TECHNICAL CONSULTANT ( EES) Direction Project Management Test Laboratories NATA National Association of Testing Authorities Quality Consumer Groups Compliance (Checktest) Standards Development Round Robin Assurance Management / Relationships

12. Role of a Laboratory Accreditation Body Accreditation bodies provide a formal recognition of laboratory competence in terms of: • Staff qualification and experience; • Equipment Calibration (NML); • Adequacy of quality assurance systems; • Soundness of testing procedures; • Validity of test methods; • Traceability of measurements to national and international standards (NML); • Accuracy of recording and reporting procedures; and • Suitability of testing facilities. 

13. Why laboratory accreditation is important to a Standards & Labelling program • Governments generally do not have the resources to monitor laboratory standards directly • Compliance programs rely on a high degree of laboratory integrity if they are to appear credible to industry, consumers and where necessary courts • A Government’s case for regulation will depend partially on the credibility of its standards development testing program • Industry often relies upon accredited laboratories to benchmark their own in-house facilities

14. Australia’s Accreditation Body • The National Association of Testing Authorities- Australia's Government-endorsed national provider of accreditation for laboratories and similar testing facilities. • NATA uses international assessment criteria - ISO/IEC 17025, plus NATA’s own rules • Member of: • APLAC Asia-pacific Laboratory Accreditation Co-operation • ILAC – International Laboratory Accreditation • NATA holds numerous mutual recognition agreements • NATA labs are used exclusively for standards development and compliance programs in Australia

15. Developing Standards & Testing CapacityA Case Study

16. Developing Standards & Capacity –case study Refrigerated Display Cabinets: MEPS 2004 In Early 2002 AU began upgrading AS 1731-2000 in preparation for MEPS regulations in 2004 The Hurdles • Lack of experience in independent test facilities in AU • ISO filler packs very expensive • ISO M packs from OS deteriorate • ISO M packs not certified • No suppliers of suitable door closers • Levels of reproducibility unknown • Appropriate MEPS levels undetermined M Packs – After 9 Months

17. Developing Standards & Capacity –case study Test Rooms Developed Test Packs Developed Filler and Measurement

18. Developing Standards & Capacity –case study Measurement Pack Calibration facility Developed

19. Developing Standards & Capacity –case study Comparative Study Undertaken on Test Packs Australian Packs ISO Packs 11.43kWh (-1.4%) 11.59kWh

20. Developing Standards & Capacity –case study Door Closer mechanisms and software

21. Developing Standards & Capacity –case study Limit of Reproducibility determined via R Robin process (+/- 2%). This then informs the compliance regime that will follow the introduction of regulations. Appropriate MEPS levels established via testing of units supplied by industry – Industry provides representative test units, government pays for testing and provides supplier with report. Results inform MEPS level

22. Compliance Program(Checktesting)

23. Compliance Programs Procedural Basis: Administrative Guidelines For The Appliance And Equipment Energy Efficiency Program Of Mandatory Labelling And Minimum Energy Performance Standards (NAEEEC 2004) Validation Basis: Statistical Basis for the Determination of Checktesting Validity Criteria (Bartels/Harrington 2004) http://www.energyrating.gov.au/admin-guidelines.html

24. Compliance Programs Step 1 – Scoping: NAEEEC sets annual budget: $200 – 300K Project Manager (Technical consultant – EES) Selects Products using various criteria: • Previous compliance record by brand* • Volume of Sales • Particularly high or low efficiency claims* • History & Credentials of the test laboratory* • Third party referrals *National Database provides valuable input

25. Compliance Programs Step 2 – Testing: • NATA laboratories competitively tender on works • Contracts are let • Appliances purchased anonymously by test laboratories • Tests Undertaken – reports to technical consultant • Technical consultant refers failures to regulator

26. Compliance Programs Step 3 – Verification: • Laboratory facilities and test units may be inspected • Faults in test units may be rectified and re tests undertaken* • Up to 3 additional units may be tested (at suppliers expense) – Stage 2 checktesting. This may be at any NATA laboratory. *The checktestprogram is not there to provide a QA program for industry

27. Compliance Programs Step 4 – Enforcement Outcomes: • Deregistration of product • Publication of failures • Public Reporting in Annual forum • Mandatory NATA testing* • Referral to ACCC (recalls) • Future compliance focus on registrations that rely on suspect laboratories* *A checktest failure is not necessarily an indictment on the original test lab.

28. Capacity Building Incentives for bringing lab facilities on-line • Regulations create a market for test labs • NAEEEC fund tests for NATA accreditation • NAEEEC may assist with NATA witness costs • NAEEEC may offer a minimum number of checktests upon accreditation • Contracts for standards development work may be offered • Commissions for the provision of expert advice may be offered

29. Keeping them Honest • Only NATA accredited laboratories are used for compliance testing & Standards development • If issues of laboratory competence are raised in any program area, NATA is called in. NATA can withdraw accreditation • NAEEEC funded round robin programs are routinely undertaken amongst NATA laboratories • Reproducibility problems referred to NATA / Standards • Process helps to inform compliance tolerances • Round Robin test units are routinely offered to industry laboratories to assist them maintain standards

30. Keeping Them Honest

31. Keys to Success in Managing Laboratories • Participation – (via Standards committees) • Government / Regulators • Industry • Consumers • Laboratories and related experts • Quality Control – • Laboratory Accreditation scheme • Regular programme of comparative testing (Round Robins) • Resources – • Standards development program • Compliance program • Incentive program for laboratories • Communications Plan • Procedural rules and rules for compliance validation • Enforcement activities • Data collection – monitoring / enforcement

32. The End For further Information on the Australian System www.energyrating.gov.au