INCORPORATING MULTIPLE LINES OF EVIDENCE INTO SEDIMENT QUALITY OBJECTIVES

1. INCORPORATING MULTIPLE LINES OF EVIDENCE INTO SEDIMENT QUALITY OBJECTIVES Stephen B. Weisberg Southern California Coastal Water Research Project

2. BACKGROUND • For many years, scientist have advocated a triad approach for evaluating sediment quality • Individual lines of evidence each have potential limitations • Most applications have been site-specific and based on best professional judgment • There are many challenges in translating scientific concept into a state-wide regulatory framework • Standardizing interpretation • Ph.D. biologists vs. B.S. engineers

3. LARGEST CONCERNS YOU EXPRESSED RELATIONSHIP BETWEEN SCIENCE AND POLICY • There are a spectrum of decisions • Selecting indicators and thresholds from individual lines of evidence • Joining multiple lines of evidence to make a station assessment • Joining multiple stations to make a water body (on site) assessment • Science is about linearizing complex information • Policy is about establishing thresholds along those gradients • Policy becomes predominant at the higher levels of information • Reconsider using a linear, numerical scoring system • Implies a linearity that may not exist • Assures equal weighting among different lines or evidence • Enhance interaction with stakeholders • Integration framework is a combination of science and policy • Place greater emphasis on application guidance

4. RELATIONSHIP BETWEEN SCIENCE AND POLICY • There are a spectrum of decisions • Selecting indicators and thresholds from individual lines of evidence • Joining multiple lines of evidence to make a station assessment • Joining multiple stations to make a water body (on site) assessment • Science is about linearizing complex information • Policy is about establishing thresholds along those gradients • Policy becomes predominant at the higher levels of information Species Abundance Species Abundance

5. LARGEST CONCERNS YOU EXPRESSED APPLICATION GUIDANCE • Moving from site-based objectives to water body assessments • Selecting implementation options where there are no chemical-specific thresholds • Consider a sequential of phased sampling approach where effort is proportional to the nature of the problem • Reconsider using a linear, numerical scoring system • Implies a linearity that may not exist • Assures equal weighting among different lines or evidence • Enhance interaction with stakeholders • Integration framework is a combination of science and policy • Place greater emphasis on application guidance

6. LARGEST CONCERNS YOU EXPRESSED • Reconsider using a linear, numerical scoring system • Implies a linearity that may not exist • Assures equal weighting among different lines or evidence • Enhance interaction with stakeholders • Integration framework is a combination of science and policy • Place greater emphasis on application guidance

7. MLOE WORKPLAN BENEFICIAL USE PROTECTION CATEGORIES • Aquatic life (Infaunal) effects • Human health effects • Fish and wildlife effects • MLOE will be used within each, but independent assessments will be conducted for each beneficial use • Task 1: Define the lines of evidence that will be used for each beneficial use • Select the indicators for those lines of evidence • Task 2: Develop a scoring system for each LOE • When a single indicator is available • When multiple indicators are available • Task 3: Integrate scoring across multiple LOEs to develop a station assessment • Task 4: Develop a strategy for incomplete data • Task 5: Develop application guidance

8. MLOE WORKPLAN • Task 1: Define the lines of evidence that will be used for each beneficial use • Select the indicators for those lines of evidence • Task 2: Develop a scoring system for each LOE • When a single indicator is available • When multiple indicators are available • Task 3: Integrate scoring across multiple LOEs to develop a station assessment • Task 4: Develop a strategy for incomplete data • Task 5: Develop application guidance

9. CLASSIC TRIAD APPROACH • Score each LOE as a binary decision • This yields 8 narratively-interpretable integrated endpoints • Relies heavily on best professional judgement • Assessment tools are too crude for binary decisions • Eight endpoints don’t capture uncertainty within individual LOE

10. CLASSIC TRIAD APPROACH • Score each LOE as a binary decision • This yields 8 narratively-interpretable integrated endpoints • Relies heavily on best professional judgement • Assessment tools are too crude for binary decisions • Eight endpoints don’t capture uncertainty within individual LOE

11. ALTERNATIVE APPROACH • Create multiple categories for each line of evidence • Recognize uncertainty associated with a single threshold • Leads to many more than 8 combinations • A headache, but provides more information • Allows for assessment of endpoints that describe magnitude of effects • Also allows for endpoints that describe confidence in the result • Confidence based on level of effect or agreement among individual lines of evidence

12. AGREEMENT ABOUT CATEGORIES LIKELY ASSESSMENT CATEGORIES FOR INDIVIDUAL LINES OF EVIDENCE • Reference condition • Slight deviation from reference • Probably defined by measurement error • Moderate effect • Severe effect • Individual lines of evidence • MLOE site assessment

13. AGREEMENT ABOUT CATEGORIES • Individual lines of evidence • MLOE: Site assessment

14. POSSIBLE SITE ASSESSMENT CATEGORIES • Reference • Unimpaired • Unimpacted • Unaffected • Protected • Affected • Individual line of evidence • Impacted • Site level • Merging multiple lines of evidence • Impaired • Water body level • Merging multiple sites • Unimpacted • Likely unimpacted • Possibly impacted • Likely impacted • Clearly impacted • Inconclusive?

15. POSSIBLE SITE ASSESSMENT CATEGORIES • Unimpacted • Likely unimpacted • Possibly impacted • Likely impacted • Clearly impacted • Inconclusive?

16. CHEMISTRY: Reference Toxicity Benthos

17. CHEMISTRY: Minor Deviation Toxicity Benthos

18. CHEMISTRY: Moderate Effect Toxicity Benthos

19. CHEMISTRY: Severe Effect Toxicity Benthos

20. NUMERICAL SYSTEM • Can be simpler than look-up charts • Also can provide more gradation for prioritization or trends analysis • Assumes a linearity • Assumes an equal weighting • Assumes the stakeholders community counts it

21. MLOE WORKPLAN • Task 1: Define the lines of evidence that will be used for each beneficial use • Select the indicators for those lines of evidence • Task 2: Develop a scoring system for each LOE • When a single indicator is available • When multiple indicators are available • Task 3: Integrate scoring across multiple LOEs to develop a station assessment • Task 4: Develop a strategy for incomplete data • Task 5: Develop application guidance

22. APPLICATION GUIDANCE • Inherently a policy issue • Stakeholder advisory committee has been tasked to do this • They have formed subcommittees to develop guidance for three applications • 303d listings • Dredging decisions • NPDES permitting

23. TECHNICAL SUPPORT • Uncertainty analysis • Sampling design guidance • Phased sampling design • Guidance for identifying contaminants of concern

24. UNCERTAINTY ANALYSIS • Assessing uncertainty at the station level • Measurement variability • Assessment tool error • Defining background level of “impact” • No system is likely to have completely unimpacted sites • Identify systems (or subsystems) that are least influenced • Use project data base to create a frequency plot for those systems

25. TECHNICAL SUPPORT • Uncertainty analysis • Sampling design guidance • Phased sampling design • Guidance for identifying contaminants of concern

26. IDENTIFYING CONTAMINANTS OF CONCERN • Chemical specific thresholds • Don’t want to rely on chemistry as a sole means for impairment decision, but chemical thresholds can be useful guidance for implementation actions • Spatial gradient analyses • Probably better in concept than practice • Sediment TIES • This is a key area for scientific advance in support of management

27. MLOE WORKPLAN • Task 1: Define the lines of evidence that will be used for each beneficial use • Select the indicators for those lines of evidence • Task 2: Develop a scoring system for each LOE • When a single indicator is available • When multiple indicators are available • Task 3: Integrate scoring across multiple LOEs to develop a station assessment • Task 4: Develop a strategy for incomplete data • Task 5: Develop application guidance

28. WHICH INDICATORS? • Infaunal effects • Sediment chemistry • Sediment toxicity • Benthic infaunal assemblage • Human health • Sediment chemistry • Fish/bivalve tissue chemistry • Fish/wildlife • Sediment chemistry • Tissue exposure • Biological effects

29. MULTIPLE INDICATORS WITHINA LINE OF EVIDENCE • Multiple toxicity tests available for the site • Chronic and acute tests • Multiple ways to interpret the same data • Equilibrium partitioning vs. empirical thresholds • Several alternatives for integrating such data

30. POSSIBLE SCORING APPROACHES • Average score • Worst score • They each measure different things • Prioritizing among tests • Most sensitive test • Least sensitive test • “Best” test • Highest quality data • Some combination of frequency and severity

31. PROPOSED SCORING WHEN MULTIPLE INDICATORS ARE MEASURED

32. WHAT IF ONLY TWO LINES OF EVIDENCE ARE AVAILABLE? • Looking for a combination of concordance and magnitude • Unimpacted: No effect from either indicator • Likely unimpacted: Small effect with no indicator, but no effect for the other • Inconslusive: Large effect with one indicator but no effect for the other • Likely impacted: Some effect for both indicators • Clearly impacted: High effect for both indicators