1 / 45

The DL Blues: The Continuing Saga of Detection and Quantitation Limits

The DL Blues: The Continuing Saga of Detection and Quantitation Limits. Richard R. Rediske, Ph.D. Annis Water Resources Institute Grand Valley State University. Problems with the MDL. MDLs vary by day, lab., analyst & method Precision at low levels is usually poor

Download Presentation

The DL Blues: The Continuing Saga of Detection and Quantitation Limits

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The DL Blues: The Continuing Saga of Detection and Quantitation Limits Richard R. Rediske, Ph.D. Annis Water Resources Institute Grand Valley State University

  2. Problems with the MDL • MDLs vary by day, lab., analyst & method • Precision at low levels is usually poor • Designed for false-positives; false negatives may be reported 50% of the time • Inherent uncertainties make enforcing permit levels @ MDL values impossible

  3. Problems with the MDL • Not conservative for limiting false positives. • Since replicates often processed over short time (e.g., single batch), long-term variability (s) is under estimated. • MDL is a “prediction limit. FP error small for only one future sample. For batch of 20 samples, Probability of FP = 1 - (0.99)20 0.2. • Standard deviation (s) estimated from small set of replicates, but its variability (even over a short time) is not taken into account.

  4. Problems with the MDL • MDL depends on precision (s) only and does not take bias into account. • Positive Bias - Concentration of trace contamination in “blanks” can exceed MDL. • MDL does not minimize false negatives (even when negative analytical bias is absent) ! • Cannot reliably report non-detects as “< MDL,” contrary to EPA and DOD guidance . • FN error at MDL is at least 50%.

  5. False Negatives at MDL Limit to report non-detects FN Probability (H0:  = MDL) FP Probability (H0:  = 0) 50% 1% 0 MDL = 3 s

  6. + LC LD LQ LC Critical Level LD Limit of Detection LQ Limit of Quantitation Can’t differentiate from background False Positive Error rate too high

  7. + + LC LD LQ LC Critical Level LD Limit of Detection LQ Limit of Quantitation Can differentiate from background False Positive Acceptable

  8. - LC LD LQ LC Critical Level LD Limit of Detection LQ Limit of Quantitation False Negative error rate too high to detect with confidence

  9. - - LC LD LQ LC Critical Level LD Limit of Detection LQ Limit of Quantitation False Negative error rate acceptable

  10. P&B LC LD LQ LC Critical Level LD Limit of Detection LQ Limit of Quantitation The analyte is detectable but the quantity is uncertain

  11. P&B P&B LC LD LQ LC Critical Level LD Limit of Detection LQ Limit of Quantitation The analyte is quantifiable with a known level of Precision and Bias

  12. Current State of the Art

  13. History • Seminal work on detection and quantitation is by Lloyd Currie • Published in Analytical Chemistry in 1968 (40, p586) • Introduced terms of • “critical level” (LC), “critical value” (CRV); the “detection decision”; with a 50% confidence level • “minimum detectable value” (MDV), “detection limit” (LD) with a 99% confidence level • “determination limit”, “minimum quantifiable value” (MQV); limit of quantitation” (LOQ); commonly “quantitation limit” (LQ) required precision, accuracy, false negative error rate and qualitative identification criteria for the intended purpose.

  14. History • On December 3, 1979, EPA proposed the 600-Series organic methods (44 FR 69463) • GC/MS Methods 624 and 625 contained a “limit of detection” for each compound • In Method 624, the LOD was defined as defined as the “minimum level at which entire system must recognizable mass spectra and acceptable calibration points” • In Method 625, the LOD was defined as the “minimum level at which the analytical system must give mass spectral confirmation.” • The LOD in Methods 624 and 625 • Were estimates of the lowest level that could be measured and the basis for the minimum level of quantitation (ML)

  15. History • Method detection limit (MDL) was first published in a paper by John Glaser and others at EPA’s laboratory in Cincinnati in 1981 in Environmental Science and Technology (15, p1426) • MDL based on Currie’s work • Employs low-level spikes rather than backgrounds • Uses Student’s t-test to allow for varying number of replicates • Has remained largely unchanged since publication

  16. MDL and ML Promulgation • On October 26, 1984 (49 FR 43234), EPA promulgated: • The 600- and 1600- Series organic methods at 40 CFR 136, appendix A • The 600-Series methods contained MDLs as detection limits • Methods 1624 and 1625 contained MLs as detection limits • The MDL procedure at 40 CFR 136, appendix B • Method 200.7 for metals at 40 CFR 136, appendix C • Method 200.7 contained MDLs as detection limits

  17. WQBEL Guidance • In 1994, EPA published draft “National Guidance for the Permitting, Monitoring, and Enforcement of Water Quality-based Effluent Limitations Set Below Analytical Detection/Quantitation Levels” (WQBEL Guidance) • Guidance suggested use of the ML as a compliance evaluation threshold when the ambient water quality criterion was below the detection limit of the most sensitive analytical method • Concerns • Industry - ML too low • States - ML not as protective as the MDL. • EPA did not complete the WQBEL Guidance, but it precipitated further discussions of detection and quantitation limits

  18. Refinement of ML • In support of the draft WQBEL Guidance, EPA refined the ML to be consistent with the ISO/IUPAC LOQ (at the time) and the ACS LOD • ML = 10 times the standard deviation of 7 replicates • ML = 3.18 times the MDL

  19. ASTM IDE and IQE • In the mid- to late- 1990s, Robert Gibbons, David Coleman, Nancy Grams, and others worked through IIAG and within ASTM Committee D19 to develop the “interlaboratory detection estimate” (IDE) and “interlaboratory quantitation estimate” (IQE) • IDE and IQE used a model of standard deviation or relative standard deviation (RSD) as a function of concentration as the basis for establishing detection and quantitation estimates

  20. EPA Method 1631 for Mercury • On June 8, 1999 (64 FR 30417), EPA promulgated Revision B to EPA Method 1631 for determination of mercury by atomic fluorescence • The Alliance of Automobile Manufacturers, Inc. and others brought suit over, among other things, the detection and quantitation limits in EPA Method 1631 and how they were developed

  21. Settlement Agreement • In Clause 6 of a Settlement Agreement signed October 19, 2000, EPA agreed to: • Re-assess procedures for determining detection and quantitation limits • Peer review the re-assessment • Provide opportunity for comment on the re-assessment • Publish a notice and invite comment on the re-assessment by February 28, 2003 • Publish a notice of final action on the re-assessment by September 30, 2004

  22. Evaluation of Concepts • Evaluation criteria for detection and quantitation • The concept must be scientifically valid • It can and has been tested • It has been subjected to peer review and publication • It is supported by a well-defined procedure • It has been accepted by the scientific community • The error rate is known or can be estimated • The concept must include routine variability • The concept must be applicable in a single laboratory • The detection limit concept should identify the concentration at which there is 99% confidence that the analyte is present • The quantitation limit should identify the concentration at which the reliability of the measurement is consistent with the capabilities of a method practiced by an experienced staff in a well-operated laboratory

  23. Withdrawal of Proposed Changes to CFR • EPA published a revised procedure for comment in 2003 • EPA received 136 comments that they could do better • Withdrawal of proposal in November 2004 • EPA announced plans to work with stakeholders to address concerns about calculation and use of detection and quantitation limits in CWA programs • This action did not change the way EPA calculates MDLs and MLs

  24. Federal Advisory Committee Formation • On January 26, 2005, EPA held a public meeting to report findings in the “Situation Assessment Report on Detection and Quantitation Approaches and Uses in [EPA’s] Clean Water Act Programs” and formation of a Federal Advisory Committee • On May 24, 2005 (70 FR 29743), EPA announced the first meeting of the “Federal Advisory Committee on Detection and Quantitation Approaches and Uses in Clean Water Act (CWA) Programs”

  25. Is this too long?

  26. FAC-FACDQ http://www.epa.gov/waterscience/methods/det/ Four Members from each Group • Environmental Labs • Industry • Water Utilities • States • Environmental Groups (One resigned) One EPA Member Consensus is defined as all members agree or not opposed

  27. Key Issues • Procedure vs method • Many regulatory standards near or below detection limits (PCBs, Mercury, TCDD, PPCPs) • Lab credibility often associated with detection limits (Mine is lower than yours) • Maze of tests and matrices make it difficult to find a universally applicable technique • Calculating detection limits can be expensive

  28. Key Issues • The procedures do not incorporate and apply Data Quality Objectives for bias, precision, representativeness, and comparability for lab and method performance at the detection and quantitation limits used in CWA programs, at all levels and frequencies of operations that can influence data use and interpretation relative to detection and quantitation limits.

  29. Accomplishments Pilot Study • Three detection and three quantitation procedures were studied • Detection • ACIL MDL • USEPA OGWDW Hubaux-Vos Yc • ASTM Yc/Lc (from IDE procedure) • Quantitation • ACIL ML • USEPA OGWDW LCMRL • ASTM IQE (at 20% and 30% RSD) • Five methods studied • Inorganic ions by EPA Method 300.0 (ion chromatography) • Total cyanide by spectrophotometry (EPA Method 335.4) • Metals by EPA Method 200.7 (ICP-OES) • Organochlorine pesticides by GC/ECD (EPA Method 608) • Semivolatile organics by GC/MS (EPA Method 625) • Between 6 and 8 labs performed Pilot Study analyses for each method

  30. Accomplishments Single Lab Procedure • Modified ACIL Procedure • Demonstrates the laboratories performance at a specified level. • Determines the lowest possible value achievable by the laboratory while meeting the Measurement Quality Objectives. • Recognition of Method Types Censored method – methods that produce no quantitative response below a certain signal threshold (Chromatographic methods) Uncensored method – methods that produce a quantitative response for each measurement regardless of concentration (ICP)

  31. Single Lab Procedure • Does the procedure provide that qualitative identification criteria defined in the analytical method are met at the determined detection and quantitation limits? • Yes qualitative identification criteria are required to be met for any results above the DL • Does the procedure adequately represent routine variability in lab performance? • Yes, the procedure uses routine method blanks and spikes generated over a period of time. • Does the procedure perform on-going verification of estimates? • Yes, both false positives (through blanks) and false negatives (through spikes) are checked and the DL and QL are adjusted if the rates are too high

  32. Single Lab Procedure • Does the procedure use only data that results from test methods conducted in their entirety? • Yes, this is explicitly required • Does the procedure explicitly adjust or account for situations where method blanks always return a non-zero result/response? • Yes, the mean blank value is added to the DL estimate • Does the procedure capable of calculating limits using matrices other than lab reagent grade water? • Yes, it is straightforward to apply the procedure to other matrices, and there is a blank/QL spike check incorporated into the procedure for individual matrices such as a specific wastewater.

  33. Single Lab Procedure • Does the procedure assess multi-laboratory and inter-laboratory variability when data from more than one lab is used? • In a multi laboratory setting, the QL would be set at a level achieved by a specified proportion of the participant laboratories • Is the procedure applicable to all users and test methods? • Yes, we believe so, any test method for which spiking is feasible • Is the procedure clearly written with enough detail so that most users can understand and implement them? • We believe so – the procedure is similar to that used in the pilot study

  34. Single Lab Procedure • Is the procedure cost effective? • Yes – the procedure is more expensive than a MDL that is only performed once, but less expensive than a MDL that must be repeated each year. In addition, good QL level bias and precision information is obtained. • For uncensored tests, easy to collect the blank data for DL • Existing MDL data can be used for the initial estimate of QL (and censored method DL) • Ongoing periodic spikes required, but • No need to start over every year

  35. Single Lab Procedure Vote on Procedure • Vote: 14 Agree, 1 Not Opposed, 5 Disagree • Not Approved • States: 4 Agree • Labs: 3 Agree, 1 Disagree • Industry: 4 Agree • Public Utilities: 2 Agree, 1 Not Opposed, 1 Disagree • EPA: 1 Disagree • Environmental Community: 1 Agree, 2 Disagree

  36. Measurement Quality Objectives • The FACDQ recommends that EPA establish quantitative MQO bounds for relevant Data Quality Indicators (DQIs) that define Quantitation for intended CWA uses. These bounds would be offered for public comment by EPA. • Vote: 9 Agree, 7 Not Opposed, 1 Disagree, 1 Absent • Not Approved • Precision ≤ 30% RSD • Accuracy (measured as recovery for single determination) = 20-180% • False Negative rate ≤ 10% • Ratio of Accuracy to Precision must be no less than 1.0 • Example: 40% Recovery / 20% RSD = 2 O.K., • Example: 20% Recovery / 30% RSD = .66 Not Acceptable • Vote: 12 Agree, 5 Not Opposed, 1 Disagree, 1 Absent • Not Approved

  37. EPA use the Data Quality Objective process to set target Measurement Quality Objectives for National Quantitation Limits for use in NPDES permit compliance testing. • A minimum of 6-7 labs be used to set National Quantitation Limits. • Data be collected, at a minimum, over 3- 6 months. • A minimum of 20 spikes be used in the calculation of each Laboratory Quantitation Limit. • The data and lab be evaluated for validity prior to acceptance. • An appropriate outlier test then is applied to the dataset. • The data are evaluated for normality, using standard statistical tests. • If the data are normally distributed then calculate the upper 95% confidence limit, which becomes the Quantitation Limit. • If the data are non-normally distributed then the 95th percentile of the Laboratory Quantitation Limit data becomes the Quantitation Limit. • Vote: 9 Agree, 8 Not Opposed, 1 Disagree,

  38. December 2008 • EPA proposes rule. Assuming a FACDQ recommendation that EPA accepts, propose to amend • Part 136 (Analytical Methods) to add new approach(es). • Part 122 (EPA Administered Permit Programs: The National Pollutant Discharge Elimination System) to add uses provisions. • EPA • Takes comments and continues rulemaking accordingly • Conducts training/outreach for states, permittees, labs • States • Comments on proposed EPA rules • Participates in EPA training/outreach • Continues existing approaches or transitions • Begins planning if federal rules are implementable with or without state rules • Permittees and Labs • Comments on proposed EPA rules

  39. December 2009 • Final rule. • All DLs and QLs promulgated after this date would be required to use the new approach(es). All previously promulgated MDLs or MLs would still be valid unless re-promulgated using the new approach(es). Preamble to this final rule could contain guidance to stakeholders or it could be a separate document issued at the same time. Rules should contain dates by which entities need to have accomplished certain tasks • Time lag for states to modify rules to fully implement regulations • Labs prepare to implement the new procedures • EPA • EPA publishes final rule and announces effective date ( ) days before actual effective date • Implements new rules or oversight of states where delegated • Begins promulgation of National Quantitation Limits based on priority

  40. December 2009 • States • Begin implementing federal rules or begin state rule promulgation (if rules are necessary, it may take another year or two) • May need to maintain duplicate system for methods/analytes with National QLs versus those without • Plan training/outreach to permittees • Permittees • Newly issued permits may specify procedures to be used to set DL and QL and other steps resulting from new rules • Existing permits may be modified by states to contain new procedures • Existing permits may automatically signal changes because of language that anticipates rule-making • If state does not modify permits or have automatic change language in permits, some permits could go 5 years under the current requirements • Laboratories • Begin using new procedure • May need to maintain duplicate procedures or nomenclature

  41. Future December 2010 • Date by which time labs must have generated QLs and DLs using the new procedure December 2011 • Date by which time delegated states must have fully implemented procedures that comply with federal requirements

  42. We must chose wisely!

More Related