Calibration Procedures and Regulatory Updates from the Laboratory Perspective

1. Calibration Procedures and Regulatory Updates from the Laboratory Perspective Charles Carter, Ph.D. Vice President of Quality and Technical Services September 21, 2010

2. Disclaimer • The title is inaccurate! • It was created a few weeks before the presentation

3. Topics • Calibration in environmental analysis • Review of linear regressions, weighted, unweighted, forced and “not forced” • Impact of calibration model on data • Evaluation of calibrations (Method Update Rule) • Effective communication between the field and the laboratory • Examples to the contrary • Brief economic update

4. Calibration Models • The relationship between the results of an MDL study and actual day to day or sample to sample sensitivity is tenuous at best. However, the impact of calibration models on the ability to detect analytes is substantial. Nonetheless, use of the appropriate calibration models is poorly understood and poorly controlled, and in many cases we are instructed to use calibration models that produce both false positives and false negatives.

5. A Data Set

7. (X,Y)

8. Minimize sum of squared deviations Σ [(predicted – actual)2] } } } r2 = 0.993 } } }

9. A Data Set (response – 14.2) / 10.3 = Conc (ug/l) (10.8 – 14.2) / 10.3 = -0.58 ug/l Absolute error 1.58 ug/l, relative error 158%

10. r2 = 0.986 Max %D = 8.8%

11. The impact of forcing • Fit might be slightly worse • Only appropriate when curve comes close to origin • A positive instrument response will produce a positive result • Sometimes clearly the best choice

12. Weighted Regressions • Unweighted Σ [(predicted – actual)2] • 1/X Weighting Σ [[(predicted – actual)2/ conc]] • 1/X2 Weighting Σ [[(predicted – actual)2/ (conc)2]] • Tend to minimize relative error as opposed to absolute error

13. Minimize sum of squared weighted deviations Σ [[(predicted – actual)]2/(conc)] r2 = 0.990 Max % D = 11%, low point 0.75%

14. The impact of weighting • Line largely defined by population midpoint and the low point on the curve • Error is minimized at the low end of curve • Higher error at high end of curve • With 1/X weighting the low point has 100x the impact of the high point on the slope. • With 1/X2 it has 10,000 times the impact.

15. Impact of calibration model on results • Two examples • GC/MS calibration data • ICP/MS continuing calibration blank data

16. GC/MS data • Calibration curve calculated three ways • Average response factor • Unweighted linear regression • Weighted linear regression • All compounds pass method linearity criteria using all three calibration models

17. The question • If a sample gave the exact same response as the lowest standard on the curve, how close would the result be to the lowest standard?

18. The results

19. ICP/MS data • Calibration curve calculated two ways • Unweighted linear regression • Weighted linear regression • The test • If the CCB result is greater than the MDL, you have a high risk of false positives • If the CCB result is less than the negative value of the MDL, you have a high risk of false negatives

20. CCB 1

21. CCB2

22. CCB 3

23. CCB 4

24. Weighted versus unweighted • Unweighted – >50% fail test. CCB results are either > MDL or < -MDL • Weighted – 1.2% fail test. One high result for molybdenum • Same data, same instrument same sensitivity, only difference is calibration model

25. Calibration models and calibration acceptance criteria • The widespread use of the unweighted linear regression in environmental analysis introduces significant error at low concentrations • The acceptance criteria for calibrations clearly are not sufficient to protect against large relative errors at low concentrations

26. The Calibration Curve that Can’t Fail! (A Digression) • “We really want to make sure we carefully define the low end of the curve.”

27. Population mean

28. Reporting limit corresponds to the low point on the curve Population mean is very near low end of curve. Irrespective of results, the line is close to the data points.

29. Method Update Rule • Introduction of Relative Standard Error to evaluate calibration curves • Dr. Dennis Edgerly and Dr. Richard Burrows

30. Example RSE calculation

31. Which works better? • Process calibration curves with various models • Linear unforced • Linear forced • 1/X weighted • 1/X2 weighted • Evaluate using correlation coefficient and RSE

32. Method 300 example

37. Barriers to Adoption of RSE and Weighted Regressions • Default instrument software setting is unweighted • Some instrument software incapable of weighting • RSE not included in instrument software • Data validator’s lack of familiarity with weighted regressions and RSE • Absence of calculation tools to assist data validators

38. Effective transfer of contractual requirements and field information • Substantial daily effort • Most common cause of significant project errors • Some examples

39. An almost ideal case

40. Too generic

41. Please fill in the blank

42. 2 page chain of custody

43. Now what?

44. Extra analyses requested

45. Nice uncluttered appearanceShort on specifics

46. Some truly flawed instructions • Tributyl tin analysis • New acquisition • Performing tributyl tin analysis • $15 • Send in QA! • The laboratory approach – “let’s pretend that tributyl tin is the same as total organic tin” • Procedure • Extract water sample with solvent • Discard solvent!!!!!!! • Analyze tin in extracted water sample

47. Ann Arbor PressDecember 26th CORRECTION The recipe for Christmas Plum Pudding in “Party Cooking”, December 24th should call for two cups currants, not two pounds ground meat.

48. Economic outlook • This year’s overall outlook • Continued gradual economic recovery with employment lagging overall economic activity • Cautious consumer spending with increased savings rate and reduction of credit card and consumer debt • Uncertainty in continued recovery due to state budget crises • Tax policy questions

49. Laboratory industry economics • Some recovery in demand in 2010 – sector specific • Partially due to stimulus • 2009 ARRA YTD = 22K • 2010 ARRA YTD = $4M • 50 to 60 jobs • Partially due to disasters • Partially due to general increases in demand • Pricing pressures still intense • Supply exceeds demand • Attempts to displace incumbent labs • Expect gradually increasing demand • Potential downturn after ARRA

50. Summary • Regression curves and the data quality concerns with unweighted regressions • Relative standard error as a preferable approach to evaluation of calibrations • Effective information transfer from field sampling activities • Examples • Brief economic update