Proposed Modifications to Method Detection Limit Determinations as Required by 40 CFR Part 136

1. Proposed Modifications to Method Detection Limit Determinations as Required by 40 CFR Part 136 Richard Burrows FSEA October 2007 ©2007, TestAmerica Analytical Testing Corp.  All rights reserved. TestAmerica & Design TM are trademarks of TestAmerica Analytical Testing Corp.

2. Topics • What is wrong with the MDL? • What is the FAC? • How was the new procedure developed? • What does it look like? • What are the next steps?

3. MDL Assumptions • Sample standard deviation represents the population standard deviation • Blank results are centered around zero • Variance is constant in the range zero – MDL spiking level • Qualitative identification is possible Unfortunately these assumptions are not valid

4. What if the blanks have positive bias? Distribution of blanks Distribution of MDL replicates 0 MDL

5. What if the blanks have positive bias? Distribution of blanks Distribution of MDL replicates MDL should be here 0 MDL

6. What if the long term standard deviation is greater than the short term? Distribution of blanks Distribution of MDL replicates MDL should be here 0 MDL

7. What if qualitative identification is a problem? Distribution of spikes at QL Distribution of blanks MDL QL 0

8. Routine measurement results

9. Federal Advisory Committee on Detection and Quantitation • The objective of this FACDQ is to provide advice and recommendations on approaches for the development of detection and quantitation procedures and uses of these procedures in CWA programs. 1st FAC meeting June 2005 11th and final FAC meeting December 2007

11. Principles for Detection and Quantitation • The detection limit procedure must: • Control false positives and false negatives • Consider the blank • Incorporate long term variability • Include a demonstration of qualitative identification capability • The Quantitation Limit Procedure must: • Incorporate a measure of accuracy and precision

12. Secondary criteria • Not too expensive • For individual lab use • Determined or estimated quickly • Fairly straightforward

13. Current cost of MDL studies • One lab – 518 active MDLs • Cost Approx $500K • TestAmerica cost • Approx $10 million Semivolatiles GC/MS 18 MDLs due to different methods, matrices, incompatible analytes Average 2 levels for each 8 instruments 18 x 2 x 8 = 288 MDL studies 288 x 7 replicates = 2016 analytical runs

14. Detection limit drivers • Two things can drive the detection limit 1. Variability of the blank Typical examples in environmental analysis include ICP, gravimetric methods Characterized by blanks with numerical results “Uncensored methods”

15. Detection limit drivers • Two things can drive the detection limit • Ability to Qualitatively identify analytes • Typical environmental examples are GC and GC/MS • Is the peak above the threshold? • Is the peak integrated? • Are the qualifier ions present and integrated? • Are the ion ratios correct? • Characterized by blanks that are usually ND

16. Use of Blanks • If we have results for method blanks we should use them • Represents routine operating conditions (included with every batch) • No extra analytical work required • Avoids any issues with dependence of variability on concentration

17. DL Concentration at which we are fairly sure that the result is real (different from a blank) -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 DL = <X> + Ks

18. K vs t Use of t gives us a prediction interval Use of K gives us a tolerance interval Degrees of Freedom (n-1)

19. Uncertainty in the true standard deviation In a test with 20 analytes, one is statistically expected to have a true MDL more than two times the determined MDL MDL = 8.5 MDL = 4 MDL = 2.5 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 Statistical uncertainty in standard deviation with 7 replicates = 0.64 x S – 2.2 x S

20. Short and Long term variability

21. DQFAC DL/QL PROCEDURE • The procedure was developed from the ACIL procedure which was piloted for 5 methods by at least 8 labs per method. • Modifications to the ACIL procedure were designed to address shortcomings noted during the pilot study

22. Detection linits are hard work

23. DL DEFINITION • Detection Limit (DL) - The minimum result which can be reliably discriminated from a blank (for example, with a 99% confidence level). • Essentially the same definition as the MDL

24. QL DEFINITION • Quantification Limit (QL): The smallest concentration of analyte demonstrated by the laboratory to meet the required precision, accuracy, false negative error rate and qualitative identification criteria for the intended purpose.

25. GENERAL PRINCIPLES • Use blanks to define the DL if numerical results are available • Incorporate the mean of the blanks • QL is based on a spiking level – so precision and accuracy information at the QL is obtained • Calculate the lowest expected result (LER) from QL spikes to protect against false negatives • Requirement to meet a given precision and accuracy at the QL is added if defined in the analytical method

26. DL/QL PROCEDURE BASICS

27. DL LER Lowest Expected Result The LER is estimated from spikes at the QL LER = X – st s is the standard deviation of the spikes X is the mean result from the spikes t is students t for the 95% confidence level

28. ONGOING CHECKS • Data is gradually added but not discarded until over 2 years old or over 100 data points • Intent is that DLs and QLs will become reflective of routine analysis • If there are major instrumentation or process changes then start data collection over

29. ONGOING VERIFICATION Or, optional batch specific verification

30. K vs t • t – targets a false positive rate that will average 1% • K – targets a false positive rate that is almost always less than 1% • The false positive rate is lower using t, but the detection limit is higher using K. • K was used in the Pilot Study • Changed to t for the censored type data to avoid unnecessary elevation of the QL

31. ACCURACY AND PRECISION • Does the procedure provide an explicit estimate of bias at LQ for limits that must be verifiable by labs at those limits? • Yes, the spikes at the QL provide verified estimates of bias • Does the procedure provide an explicit estimate of precision at LQ for limits that must be verifiable by labs at those limits? • Yes, the spikes at the QL provide a verified estimate of precision MDL – No estimate of bias or precision at the quantitation limit

32. FALSE POSITIVE AND FALSE NEGATIVE RATES • Does the procedure provide an explicit false positive rate for blanks? • Yes, the procedure sets the DL at the level statistically predicted to be the 1% false positive level, and then verifies and corrects that level if necessary once sufficient data is available • Does the procedure provide an explicit false negative rate • Yes, the procedure requires that no more than 5% of QL level spikes return false negatives MDL – Yes for false positives (but no verification) No for false negatives

33. QUALITATIVE IDENTIFICATION AND ROUTINE VARIABILITY • Does the procedure provide that qualitative identification criteria defined in the analytical method are met at the determined detection and quantitation limits? • Yes qualitative identification criteria are required to be met for any results above the DL • Does the procedure adequately represent routine variability in lab performance? • Yes, the procedure uses routine method blanks and spikes generated over a period of time. MDL – No for qualitative id criteria, No for routine variability

34. VERIFICATION AND MATRICES • Does the procedure perform on-going verification of estimates? • Yes, both false positives (through blanks) and false negatives (through spikes) are checked and the DL and QL are adjusted if the rates are too high • Is the procedure capable of calculating limits using matrices other than lab reagent grade water? • Yes, it is straightforward to apply the procedure to other matrices, and there is a blank/QL spike check incorporated into the procedure for individual matrices such as a specific wastewater. MDL – No for ongoing verification, Yes for matrices other than reagent water

35. COMPLETE METHOD AND NON-ZERO BLANKS • Does the procedure use only data that results from test methods conducted in their entirety? • Yes, this is explicitly required • Does the procedure explicitly adjust or account for situations where method blanks always return a non-zero result/response? • Yes, the mean blank value is added to the DL estimate MDL – Yes for entire methods No for method blank non-zero adjustment

36. EASE OF USE AND COST EFFECTIVENESS • Is the procedure clearly written with enough detail so that most users can understand and implement them? • We believe so – the procedure is similar to that used in the pilot study • Is the procedure cost effective? • Yes – the procedure is more expensive than a MDL that is only performed once, but less expensive than a MDL that must be repeated each year. In addition, good QL level bias and precision information is obtained. MDL – In principle the MDL is clearly written, but there are many differences in interpretation MDL cost effectiveness is debatable

37. MULTI-LAB AND GENERAL APPLICABILITY • Does the procedure assess multi-laboratory and inter-laboratory variability when data from more than one lab is used? • In a multi laboratory setting, the QL would be set at a level achieved by a specified proportion of the participant laboratories • Is the procedure applicable to all users and test methods? • Yes, we believe so, any test method for which spiking is feasible MDL – Yes to these questions

38. What is this like in practice • Labs will have a population of method blanks • If numerical results, calculate X +Ks = DL • Labs will have a set of spike results at of below the expected QL from MDL studies • Use the spiking level as the initial QL estimate • Calculate recovery, RSD, and LER • Check to see if QL meets criteria

39. Method 625

40. 200.8 initial DLs

41. 200.7 Initial DLs

42. Modifications from the MDL • Blanks are used instead of spikes to set the DL where possible • Spikes are required at the QL so that precision and accuracy at the QL are known • The mean of the blanks is incorporated in the DL so that blank bias is accounted for • Emphasis is on data collected over time so that routine performance is reflected • False negatives are controlled

43. December 2007 • FACDQ consensus report on approach(es) and uses • EPA • Confirmatory testing of consensus approach(es) • Begins rule-making process • Begins planning training/outreach activities • States • Continue existing approaches for handling D/Q issues or begins to transition as the states choose • Permittees • No change in permits unless states begin to react/transition • Laboratories • No change unless states begin to react/transition

44. December 2008 • EPA proposes rule. Assuming a FACDQ recommendation that EPA accepts, propose to amend • Part 136 (Analytical Methods) to add new approach(es). • Part 122 (EPA Administered Permit Programs: The National Pollutant Discharge Elimination System) to add uses provisions. • EPA • Takes comments and continues rulemaking accordingly • Conducts training/outreach for states, permittees, labs • States • Comments on proposed EPA rules • Participates in EPA training/outreach • Continues existing approaches or transitions • Begins planning if federal rules are implementable with or without state rules • Permittees • Comments on proposed EPA rules • Still no change in permits, unless… • Laboratories • Comments on proposed EPArules • Still no change unless…

45. December 2009 • Final rule. • All DLs and QLs promulgated after this date would be required to use the new approach(es). All previously promulgated MDLs or MLs would still be valid unless re-promulgated using the new approach(es). Preamble to this final rule could contain guidance to stakeholders or it could be a separate document issued at the same time. Rules should contain dates by which entities need to have accomplished certain tasks • Time lag for states to modify rules to fully implement regulations • Labs prepare to implement the new procedures

46. December 2009 (continued) • EPA • EPA publishes final rule and announces effective date ( ) days before actual effective date • Implements new rules or oversight of states where delegated • Begins promulgation of National Quantitation Limits based on priority • States • Begin implementing federal rules or begin state rule promulgation (if rules are necessary, it may take another year or two) • May need to maintain duplicate system for methods/analytes with National QLs versus those without • Plan training/outreach to permittees

47. Dec 2009 (continued) • Permittees • Newly issued permits may specify procedures to be used to set DL and QL and other steps resulting from new rules • Existing permits may be modified by states to contain new procedures • Existing permits may automatically signal changes because of language that anticipates rule-making • If state does not modify permits or have automatic change language in permits, some permits could go 5 years under the current requirements • Laboratories • Begin using new procedure • May need to maintain duplicate procedures or nomenclature

48. Is it time to retire yet? December 2010 • Date by which time labs must have generated QLs and DLs using the new procedure December 2011 • Date by which time delegated states must have fully implemented procedures that comply with federal requirements

49. SUMMARY • DQ/QL procedure • Uses long term data • Takes account of blank bias • Considers qualitative identification • Checks actual performance against the calculated limits to accommodate non-normal data • Develops precision/accuracy information at the QL Whatever the DL/QL procedure, careful selection of the appropriate calibration model is vital to achieve accurate quantitation at low levels

50. How much work is this? • For uncensored tests, easy to collect the blank data for DL • Existing MDL data can be used for the initial estimate of QL (and censored method DL) • Ongoing periodic spikes required, but • No need to start over every year