ASTM International Proficiency Test Programs and Interlaboratory Studies Program Presenters: Steve Hepburn, PTP Dire

1. ASTM International Proficiency Test Programs and Interlaboratory Studies ProgramPresenters:Steve Hepburn, PTP DirectorPhillip Godorov, ILS Director

2. ASTM Proficiency Testing Programs What is a Proficiency Testing Program? • Designed as a statistical quality control tool enabling participating laboratories to assess their performance in conducting ASTM or other test methods • Uniform samples are analyzed by participating labs using specified methods and results are reported for statistical treatment September 2012

3. ASTM Proficiency Testing Programs ASTM : • Provides management and administrative support to the program • Program registration, contract negotiations for sample prep and distribution, data collection and generation of statistical summary reports • Coordinates the preparation and distribution of test samples • Sample are prepared by outside contractors • Provides reporting instructions, worksheets and electronic data report forms for submitting data September 2012

4. ASTM Proficiency Testing Programs • ASTM statistical summary reports include: • Listing of all lab data • Charts and graphs • Lab data is coded to maintain lab confidentiality. • Final summary reports are provided electronically to participants and appropriate Committee contacts. • ASTM Custom Company Reports – provided to companies with multiple lab performance data upon request. September 2012

5. ASTM #2Diesel Fuel Sample ID: DF21102 Acid Number – D664 Lab data coded to maintain lab confidentiality September 2012

6. ASTM Proficiency Testing Programs September 2012

7. ASTM Proficiency Testing Programs Box and Whiskers graph to compare methods September 2012

8. ASTM Proficiency Testing Programs • Globally recognized programs: • Over 4,300 laboratories participating • Over 57% participation outside continental U. S. September 2012

9. ASTM Proficiency Testing Programs www.ASTM.org Click here

10. ASTM Proficiency Testing Programs Drop down for complete listing

11. ASTM Proficiency Testing Programs Metals Testing: (E28 and E01) • Mechanical Properties Testing of Metals • Plain Carbon and Low-Alloy Steel (Chemical Analysis) • Stainless Steel (Chemical Analysis) • Aluminum (Chemical Analysis) • Determination of Gold in Bullion Plastics Testing: (D20) • Polyethylene Plastics Testing • Multiple Plastics - Mechanical Properties Testing • Polypropylene - Mechanical Properties Testing • Flammability of Plastics Testing • Thermal Analyses of Plastics • Elemental Analyses of Plastics ASTM PROGRAMS September 2012

12. ASTM Proficiency Testing Programs Aromatic Hydrocarbons Testing (D16) Engine Coolants Testing (D15) Electrical Insulating Liquids: (D27) • Insulating Fluid Quality • Dissolved Gas Analyses-In-Service Insulating Fluid Textiles :(D13) • Light, Medium, Heavy Fabric Testing • Yarns and Threads ASTM PROGRAMS September 2012

13. ASTM Proficiency Testing Programs 24ASTM PROGRAMS Petroleum Products and Lubricants (D02) Turbine Oil Lubricating Grease Crude Oil In-Service Diesel Oil In-Service Hydraulic Fluids / Oils Base Oils Wax General Gas Oils Ultra Low Sulfur Diesel Fuel Ethanol Biodiesel(B100) Fuel Petroleum Coke Engine Oil Lubricants #2 Diesel Fuel Aviation Turbine (Jet A) Fuel Motor Gasoline #6 Fuel Oil Automatic Transmission Fluid Gear Oil Reformulated Gasoline Hydraulic Fluids & Oils Lube Additives Industrial Gear Oil Naphtha September 2012

14. ASTM Proficiency Testing Programs ASTM's Cement and Concrete Reference Laboratory (CCRL) • Works with • ASTM Committee C01 - Cement • ASTM Committee C09 - Concrete and Aggregates • Located in Frederick, MD • Operates the following proficiency sample programs: • Portland cement • Hydraulic cement concrete • Masonry mortar • Blended cement • Pozzolans • Masonry cement • Concrete masonry units Visit the CCRL web site on ASTM web page for more information.

15. Use of Programs by Participants • Lab QA Tool • Data for monitoring lab strengths and weaknesses • Residual sample material useful as internal QA sample • Tool to evaluate other test methods for lab use • Participation helps satisfy lab accreditation requirements • Can demonstrate testing capability to customers • Results and associated statistics helps ASTM validate test method performance under real world conditions September 2012

16. Use of Programs by ASTM Committees • Obtain valuable feedback on methods • Resource for increased membership • Database of laboratories for test method validation September 2012

17. Establishing New Programs • Concept approved by Executive Subcommittee • Program pre-proposal • Market research - survey • Final program proposal • Bulk sample requirements • Sample preparation and distribution • Promote & Launch ASTM staff will provide guidance and assistance. September 2012

18. ASTM Proficiency Testing Programs The programs are open to all companies and industry test professionals worldwide. September 2012

19. ASTM Proficiency Testing Programs For more information please contact: ASTM - PTP Steve Hepburn shepburn@astm.org Phone: (610) 832-9688 “Until a measurement is in a state of statistical control, it can not be believed in any logical sense that it's measuring anything at all."  Dr. Churchill Eisenhart September 2012

20. ASTM International Interlaboratory Study Program (ILS)Presenter:Phillip Godorov, ILS Director September 2012

21. The Interlaboratory Study (ILS) ProgramWho we are and what we do … We were originated following approval by the Board of Directors to create a unit that would help to strengthen the perceived quality of ASTM Test Methods by: • facilitating the production of data to develop Precision & Bias statements and Research Reports; • providing administrative support to the committees; • offering financial support to the committees; • helping to ensure the confidentiality of participating labs. September 2012

22. Administrative Support • Review of Experimental Design • Assistance identifying volunteer laboratories • Identification of sample vendors • Coordination of sample distribution • Data collection • Statistical processing • Generation of reports September 2012

23. Strengths of the ASTM Interlaboratory Study Program • Input is encouraged from active Committee volunteers, as well as non-members, broadening the range and diversity of the study participants, allowing the study to most accurately demonstrate expected “real-world” precision • Scientific neutrality of ASTM in reviewing test data • Providing value-added Quality Assurance Programs to participating laboratories September 2012

24. Potential Benefits of Participation • Statistical program to monitor strengths and weaknesses of lab testing when compared to peers • Assess testing performance and adherence to written procedures by lab technicians • Recognition in the final Research Report • Upon publication of the results of an ILS, all participating labs are eligible to receive a free 12 month listing in ASTM’s Lab Directory September 2012

25. Benefits to the Committees • Obtain valuable feedback on methods, leading to the correction of errors and omissions, as well as highlighting the need for technical updates • Resource for increased membership • Database of laboratories for test method validation • Database of suppliers for common study materials September 2012

26. Establishing New Programs • Concept registered as a Work Item • Program registered online as an ILS • Initial conference call, with the a technical contact from the committee, to establish the basic study parameters • Experimental design (with input from the committee’s statistical support person, if available) • Identification of study materials, suppliers, a distributor, and volunteer laboratories September 2012

27. Administering Programs • Conference Calls or Virtual Meetings with the participants to discuss specific study instructions • Coordination of acquisition and distribution of study material • Physical • Electronic • Collection of data • Analysis of data • Compilation of Precision Statement and Draft Research Report (from registered information) September 2012

28. Precision • To calculate precision, we need usable data from at least six laboratories. • The closer that we can get to 30 participants, the better. • The precision statement in an ASTM test method is not meant to qualify it as good or bad. • The published precision is there to help a user of the standard understand what can be expected based on the real world results of others. September 2012

29. The Statistics (in a nutshell) ~ ASTM E691 • All of the test results from one laboratory on one material constitute a cell. • The cell average is calculated for each laboratory. • The standard deviation of the test results in each cell is calculated. • Then we calculate the average of all the cell averages for any one material – this shows up in the P&B as the “average.” • For each laboratory we calculate the cell deviation by subtracting the cell average from the average of all the cell averages. • Finally, we calculate the Standard Deviation of the Cell Averages. September 2012

30. Repeatability (r) ranges • With 95% confidence, the same operator, in the same laboratory, using the same equipment, under the same conditions, should obtain results when testing the same material that agree within this range. • Example: repeatability range = 2.4 ppm • Result 1: 79.1 ppm Result 2: 81.6 ppm • Results differ by 2.5 ppm, therefore: Suspect • Internal laboratory investigation may be advisable September 2012

31. Reproducibility (R) ranges • With 95% confidence, two operators, in different laboratories, using different equipment, under conditions meeting those specified in the standard, should obtain results when testing the samematerial that agree within this range. • Example: reproducibility range = 3.2 ppb • Result from lab 1: 50.8 ppb • Result from lab 2: 47.9 ppb • Results differ by 2.9 ppb, therefore: As Expected September 2012

32. Bias • To calculate bias, we need to include a reference “standard” among the sample specimens distributed to the participating laboratories. • Bias may be determined as the average discrepancy between the “known” value and the reported values. September 2012

33. Precision and Bias September 2012

34. Remember … • You can correct for bias. • You cannot correct for imprecision. • An ILS can be used to demonstrate improvement as standards are modified. • Compare results from Method 1 with those from Method 2 • The presence of a Precision Statement can make a standard stronger, but it doesn’t make it more repeatable. September 2012

35. Research Reports • The ASTM Form and Style Manual (Section A29.1) states, "Where numerical data have been generated to establish the precision and bias of a test method, a research report is required.” • ASTM Research Report Template • The draft research report should be made available to committee members while the associated precision statement is on ballot. • Research report numbers are assigned after the ballot is approved. September 2011

36. Parts of a Research Report • List of participating laboratories • Description of samples with their suppliers • A copy of the laboratory instructions • Description of equipment / apparatus used • All of the raw data (lab names hidden) • A statistical summary • A copy of the precision and bias statement September 2012

37. Research Report Assistance • ILS can assist with the following research report tasks: • Answer any research report inquiries • Create research reports from ILS studies • Reformat older research reports to fit the ASTM research report Template • Assign research report numbers September 2012

38. Research Report Numbers • A research report number will be assigned by ASTM when all of the following have been completed: • the research report is submitted to ILS; • it has been reviewed for completeness; • the ballot item to include the corresponding precision and bias statement is approved for publication. September 2012

39. http://www.astm.org/ILS/ ILS informational Web pages September 2012

40. www.ASTM.org September 2012

41. September 2012

42. September 2012

43. September 2012

44. If you are updating an ILS that you’ve already started When you are starting a brand new ILS September 2012

45. First Page of the Registration September 2012

46. Technical Contact September 2012

47. Analyses September 2012

48. Replicates September 2012

49. Materials September 2012

50. Suppliers September 2012