Deana M. Crumbling, M.S. Technology Innovation Office U.S. Environmental Protection Agency and

1. Modernizing Site Cleanup: Managing Decision Uncertainties Using the Triad Approach Deana M. Crumbling, M.S. Technology Innovation Office U.S. Environmental Protection Agency and Kira P. Lynch, M.S. Innovative Technology Advocate Seattle District, US Army Corps of Engineers

2. Take-Home Message • Triad Approach= Integration of systematic planning, dynamic work plans, and real-time analysis as applied to wastes and contaminated sites  time & costs;  decision certainty • Theme for the Triad Approach = Managing the largest sources of decision error, especially the samplingrepresentativeness of data

3. Talk Outline • “Modernizing” from what to what? • Broadening the “data quality” concept • Intuitive terminology for data quality • Case Study

4. Characterization & Cleanup Strategy:Where We’ve Been

5. Start here The Historical Process-Driven Approach • Easy to get lost in a maze, especially if no clear exit strategy • Sample and resample, hoping the data will tell us what to do • Reasonable for the 1970/80s • Limited experience, knowledge • Few tools available for either • data generation or cleanup • A rote one-size-fits-all process can get you through a maze; • BUT it will not be a • resource -efficient trip.

6. START: “define the nature and extent of contamination.” 1 2 One-size-fits-all lab analyses 3 2 3 2 1 We need more information REPORT 2 1 6

7. Characterization & Cleanup Strategy:Where We Are Heading

8. EXIT START Experience, Knowledge as a Foundation Key Features: • Project planning (vs. process) • Multidisciplinary team • Stakeholders involved • Create opportunities for real-time decision-making to save time and $$ • Real-time decisions need real-time data • Project-specific CSM • identifies data gaps • evolve in real-time

9. Systematic Planning Dynamic Work Plans Real-time Measurement Technologies A Systems-Approach Framework The Triad Approach

10. Unifying Concept for Triad: Managing Uncertainty Systematic planning means… • Managing uncertainty about project goals • Includes stakeholders to set priorities & select ultimate goals • Identifies decision goals w/ tolerable overall uncertainty • Identifies major uncertainties (cause decision error) • Identifies the strategy to manage each major uncertainty • Managing uncertainty in data • Sampling uncertainty: Use field methods and a dynamic work plan to effectively manage sample representativeness • Analytical uncertainty: Various strategies available to manage any residual analytical uncertainties from field methods

11. Technical Team Development • Assemble the project team by getting the right people involved • May include: statistician, chemist, hydrologist, legal or regulatory advisor, biologist, geologist, etc.

12. Prevailing wind direction Transport Medium (air) Releasemechanism(volatilization) Exposure point Inhalation Inhalation Ingestion TransportMedium(soil) Waste(Source) Water table Ground water flow Transport medium (ground water) Conceptual Site Model (CSM)

13. } Mix And Match Dynamic Work Plans • Real-time decision-making “in the field” • Evolve CSM in real-time • Implement pre-approved decision tree using senior staff • Contingency planning: most seamless activity flow possible to reach project goals in fewest mobilizations • Real-time decisions need real-time data • Use off-site lab w/ short turnaround? • Use on-site analysis? • Use mobile lab with conventional equipment? • Use portable kits & instruments? In all cases, must generate data of known quality

14. Generating Real-time Data Using Field MethodsManage Uncertainty through Systematic Planning • Need clearly defined data uses—tie to project goals • Understand dynamic work plan—branch points & work flow • Project-specific QA/QC protocols matched to intended data use • Select field analytical technologies to • Support the dynamic work plan (greatest source of $$ savings) • Manage sampling uncertainty (improves decision quality) • Select fixed lab methods (as needed) to • Manage uncertainties in field data (as ONE aspect of QC) • Supply analyte-specific data and/or lower quantitation limits (as needed for regulatory compliance, risk assessment, etc.)

15. Data Quality as a Tool to Achieve Decision Quality

16. Perfect Analytical Chemistry Non- Representative Sample + “BAD” DATA Data is Generated on Samples Distinguish: Analytical Quality from Data Quality

17. What is “Data Quality”? Data Quality = The ability of data to provide information that meets user needs • Users need to make correct decisions • Data quality is a function of data’s… • ability to represent the “true state” in the context of the decision to be made • The decision defines the scale for the “true state” • information content (including its uncertainty)

18. D E C I S I O N D E C I S I O N Goal Making The Data Quality “Chain” Sampling Analysis Sample Support

19. #2 #3 #1 Sample Support: Critical to Representativeness Sample Volume & Orientation The Nugget Effect Sample Prep Same Contaminant Mass in Nugget,but Different Sample Volumes Produce Different Concentrations The decision driving sample collection: Assess contamination resulting from atmospheric deposition 19

20. D E C I S I O N D E C I S I O N e.g., Method 8270 Goal Making The Data Quality “Chain” Sampling Analysis Extract Cleanup Method(s) Result Reporting Sub- Sampling Sampling Design Sample Support Sample Preservation Determinative Method(s) Sample Preparation Method(s) All links in the Data Quality chain must be intact for Decision Quality to be supported !

21. Partitioning Variability: Sample Location vs. Analytical Method Analytical (between methods) ~ 5% 500 On-site 416 Lab 331 On-site 286 Lab 2 7 39,800 On-site 41,400 Lab Sample Location ~ 95% 1,280 On-site 1,220 Lab 164 On-site 136 Lab 6 1 3 4 5 24,400 On-site 27,700 Lab 27,800 On-site 42,800 Lab

22. Total Uncertainty Analytical Uncertainty Ex. 1 Sampling Uncertainty 3 X Ex. 3 Ex. 2 1/3 X Ex. 1 Ex. 3 Ex. 2 Summing Uncertainties Uncertainties add according to (a2 + b2 = c2)

23. Less likely More likely Decision Quality vs. Analytical Quality ¢ ¢ ¢ ¢ ¢ ¢ ¢ $ $ $ $ $ $ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ Manylower analytical quality data points Higher information value of the data set Fewerhigh analytical quality data pointsLower information value of the data set Goal: A defensible site decision that reflects the “true” site condition

24. Less likely More likely Decision Quality vs. Analytical Quality ¢ ¢ ¢ ¢ ¢ ¢ ¢ $ $ $ $ $ $ $ $ $ $ $ $ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ NOW, few high analytical quality data pointswill Highly informative data set Many “lower analytical quality” data points Higher information value of the data set Fewerhigh analytical quality data pointsLower information value of the data set Nearly Certain Goal: A defensible site decision that reflects the “true” site condition

25. $ $ $ $ $ $ Fixed Lab Analytical Uncertainty Ex 1 Sampling Uncertainty ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ $ $ $ $ $ $ Remove hot spots Decreased Sampling Variability after Removal of Hotspots Ex 3 Fixed Lab Data Field Analytical Data Ex 2 Ex 1 Ex 2 Sampling Uncertainty Controlled through Increased Density Ex 3 Improve Decision Quality--Manage Uncertainties From This To This

26. Sample Representativeness: Key to Environmental Data Quality Finally able to address this issue defensibly and affordably! • Cheaper analyses increase sample density • Real-time analyses support real-time decision-making • Rapid feedback permits course correction and smart sampling • Focus on overall data uncertainty • Remember that analytical uncertainty is but a fraction

27. High spatial density Low DL + analyte specificity Manages sampling uncertainty Manages analytical uncertainty Definitive sampling quality Screening analytical quality Definitive analytical quality Screening sampling quality Marrying Analytical Methods to Make Sound Decisions Involving Heterogeneous Matrices Cheap screening analytical methods Costly definitive analytical methods

28. High spatial density Low DL + analyte specificity Manages sampling uncertainty Manages analytical uncertainty Collaborative Data Sets Marrying Analytical Methods to Make Sound Decisions Involving Heterogeneous Matrices Cheap screening analytical methods Costly definitive analytical methods

29. Managed sampling uncertainty: achieved very high confidence that all excessive contamination located and removed Managed analytical uncertainty as additional QC on critical samples; confirm & perfect decision levels used with field kits Case Study: Wenatchee Tree Fruit Site • Pesticide IA kits guide dynamic work plan: 56 tons soil segregated for incineration; 334 tons landfilled 230 IA analyses (2 different kits + QC) + 29 fixed-lab samples for 33 analytes • Clean closure data set • 33 fixed lab samples (floor & partial sidewall) +16 IA (sidewalls) • Demonstrate full compliance with all regulatory requirements for all • 33 pesticide analytes with >95% statistical confidence • Projected cost: ~$1.2M; Actual: $589K (Save ~ 50%) • Field work completed: <4 months; single mobilization

30. Terminology to Integrate Data Quality into Decision Quality

31. “Data Quality” Terminology Current terminology usage does not focus on the goal of decision quality • Irony: Great focus on the quality of data points; but overall quality of decisions easily unknown. • Current usage does not distinguish • Methods vs. data vs. decisions • The factors that impact each step in the process • Relationships between different aspects of quality

32. = = Methods Data Decisions Screening Methods Screening Data Uncertain Decisions “Definitive” Methods “Definitive” Data Certain Decisions The SYSTEM functions as if it believes that… Distinguish: Analytical Methods from Data from Decisions

33. Misleading Terminology Field Screening • False Implications: • All methods run in the field are screening methods. • All data produced in the field are of screening quality. • Fixed labs using definitive analytical methods don’t produce screening quality data. • Fixed labs don’t use screening methods.

34. “Effective Data” “Decision Quality Data” Data of known quality that can logically be demonstrated to be effective for making the specified decision because both the sampling and analytical uncertainties are managed to the degree necessary to meet clearlydefined (and stated) decision confidence goals

35. Wenatchee Tree Fruit Case Study: Soil Removal Using Field Analytical and a Dynamic Work Plan 35

36. Wenatchee Tree Fruit Project Overview • Action required to achieve clean closure • 390 tons of soil removed (56 tons incinerated; 334 tons landfilled) • Total cost • Projected: ~$1.2M; Actual: $589K • Savings: ~50% • Total field time • Single mobilization: <4 months from start of field work until project completion • Outcome: Happy client, regulator, stakeholders

37. Systematic Planning

38. Coordinate/Assemble Teams • Who’s Who?: Coordinate with client, regulators and stakeholders • Planning Team: client, State, stakeholder, and USACE staff • Technical/Field Team: USACE staff, prime contractor staff, and subcontractor staff • Community outreach found little additional interest

39. First Step: Identify Decisions • Problem: Pesticide contamination of vadose soil • Decisions to be made: • Locate and remove contamination • Remaining soil meet WA state cleanup stds • Manage excavated material for disposal • incineration • landfilling

40. Desired Decision Confidence • Detect contamination • Grid size set to detect a 5 ft. x 10 ft. elliptical hotspot • Remove contamination so that remaining soil meets stringent WA state regulatory cleanup standards: • for 33 individual pesticide analytes • to a 95% statistical confidence

41. State the Decision Goals (the Data Quality Objectives) • Provide results of sufficient analytical quality to • guide soil removal, • segregate and classify wastes for final disposal, and • confirm compliance with the required regulatory closure decision confidence. • Provide turnaround times for data that can support real-time decision-making in the field. • Provide sufficient sampling density to detect a 5X10 ft. hotspot. Field Analytical Field Analytical Fixed Lab Fixed Lab Field Analytical Field Analytical

42. Managing Sampling Uncertainty Understanding how contamination occurred

43. Site Grid with Probable Locations of Buried Bags Row A FR2/3 FR4/5 Row B Row C Col 1 Col 2 Col 3 Col 4 Col 5 Col 6 Col 7 Col 8 Col 9 Original Remediation Boundary X-Y Coordinate Origin Final Remediation Boundary NorthDrawing not to scale

44. Optimize the Data Collection Design • Use a Dynamic Work Plan • Use immunoassay (IA) field kits for on-site analysis to guide DWP • Perform pre-field work pilot study to • Assess IA kit suitability • Estimate field/IA kit decision/action levels • Evaluate Geoprobe performance • Prepare SOPs and contingency plans • Use fixed lab analyses to generate site closure confirmation data sets

45. Systematic Planning: Analytical Optimize On-Site Methods Pre-field work pilot study: • Compared IA to analyte-specific analyses • Understand cross-reactivity behavior of IA kits • Establish initial field decision/action levels: • 5 ppm for sum DDT; 0.086 ppm for sum cyclodienes • Project-specific SOPs established (PBMS) to improve project performance and save labor costs • Adjusted range of calibration standards • Increased the volume of the extraction solvent • Used a different solvent for the cyclodiene kit

46. Immunoassay Kits One kit for Cyclodiene Family One kit for DDT Family 46

47. Field Lab DDT,ppm Field Lab Endrin Regression Analysis (R2 = .60) Field Lab Endrin, ppm Data Comparability 47

48. QA/QC for IA Kits • 3-point calibration & CCV w/ each batch (12 samples) • Reagent blank • Matrix duplicate (intra-laboratory sample split) • LCS [prepared from a purchased (soil) PE sample] • Split sample confirmation analysis (by fixed lab analysis) for samples representing critical decision points • Excavation boundaries • Clean closure data set for regulatory compliance

49. Systematic Planning: Analytical Optimize Off-site Methods Certain fixed lab methods for pesticides were optimized using PBMS principles: • Organophosphorus (OP) pesticides: • SW-846 Method 8141 (GC/NPD) was changed to SW-846 Method 8270 (GC/MS) • Carbamates by GC: • A blend of EPA Water Method 632 and SW-846 Method 8141 (GC/NPD) was used • Paraquat in soil by spectrophotometry: • An industry developed method was used

50. Original Remediation Boundary X-Y Coordinate Origin NorthDrawing not to scale Site Characterization Sample Site Grid Showing DP Core Sampling Locations Row A FR2/3 FR4/5 Row B Row C Col 1 Col 2 Col 3 Col 4 Col 5 Col 6 Col 7 Col 8 Col 9 Final Remediation Boundary