Science and Decisions: Advancing Risk Assessment

1. Science and Decisions: Advancing Risk Assessment Risk Assessment Specialty Section (RASS) Monthly Telecon Joseph Rodricks, Environ Jonathan Levy, Harvard School of Public Health May 13, 2009

2. Study motivation • Risk assessment is at a crossroads, and its credibility is being challenged. • Science is increasingly complex. • Risk assessment is being extended to address broader environmental questions, such as life-cycle analysis and issues of costs, benefits, and risk-risk tradeoffs. • Stakeholders are often disengaged from the risk-assessment process at a time when risk assessment is increasingly intertwined with societal concerns. • Disconnects between the available scientific data and the information needs of decision-makers hinder the use of risk assessment as a decision-making tool.

3. Committee’s charge (I) • “An NRC committee will develop scientific and technical recommendations for improving the risk analysis approaches used by the U.S. Environmental Protection Agency (EPA)…The committee will consider analyses applied to contaminants in all environmental media (water, air, food, soil) and all routes of exposure (ingestion, inhalation, and dermal absorption). The committee will focus primarily on human health risk analysis and will comment on the broad implications of its findings and recommendations to ecological risk analysis. In making recommendations, the committee will indicate practical improvements that can be made in the near term (2-5 years) and improvements that would be made over a longer term (10-20 years).”

4. Committee’s charge (II) • Increased role for probabilistic analysis in risk analysis, including the potential expanded role for expert elicitation • Scientific bases for and alternatives to default assumption choices made in areas of uncertainty • Quantitative characterization of uncertainty resulting from all steps in the risk analysis • Approaches for assessing cumulative risk resulting from multiple exposures to contaminant mixtures, involving multiple sources, pathways, routes • Variability in receptor populations, especially sensitive subpopulations and critical life stages

5. Committee’s charge (III) • Biologically relevant modes of action for estimating dose-response relationships, and quantitative implications of different modes • Improvements in environmental transport and fate models, exposure models, physiologically based pharmacokinetic (PBPK) models, and dose-response models • How the concepts and practices of ecological risk analysis can help inform and improve the concepts and practices of human health risk analysis, and vice versa • Scientific basis for derivation of uncertainty factors • Use of value-of-information analyses and other techniques to identify priorities and approaches for research to obtain relevant data to increase the utility of risk analyses

6. Committee membership Thomas Burke (Chair), Johns Hopkins Bloomberg School of Public Health A. John Bailer, Miami University John M. Balbus, Environmental Defense Joshua T. Cohen, Tufts Medical Center Adam M. Finkel, University of Medicine and Dentistry of New Jersey Gary Ginsberg, Connecticut Department of Public Health Bruce K. Hope, Oregon Department of Environmental Health Jonathan I. Levy, Harvard School of Public Health Thomas E. McKone, University of California Gregory M. Paoli, Risk Sciences International Charles Poole, University of North Carolina School of Public Health Joseph V. Rodricks, ENVIRON International Corporation Bailus Walker Jr., Howard University Medical Center Terry F. Yosie, World Environment Center Lauren Zeise, California Environmental Protection Agency

7. Evaluation strategy • Committee concluded early on that risk assessment can be “improved” in two different ways • Improving technical analysis (the development and use of scientific knowledge and information to improve characterizations of risk) • Improving utility (making risk assessment more relevant and useful to risk management decisions)

8. Structure of report conclusions • Design of risk assessment • Uncertainty and variability • Selection and use of defaults • Unified approach to dose-response assessment • Cumulative risk assessment • Improving the utility of risk assessment • Stakeholder involvement • Capacity-building

9. Design of risk assessment • Design: The process of planning a risk assessment and ensuring that it has the attributes desired by the decision maker given various system constraints • Analogy to product design: What is the “right” car to buy? • From decision-support perspective, there are multiple desirable attributes which may at times conflict with one another • Use of best science and methods • Inclusiveness of scope • Inclusiveness of process • Transparency • Timeliness

10. Key conclusions • Increased attention needed to the design of risk assessment at its formative stages • Planning and scoping and problem formulation (as in EPA ecological and cumulative risk guidance) should be formalized and implemented.

11. Key design steps • Planning and scoping: Discussion among decision-makers, assessors, and stakeholders to establish issue to be assessed and goals, breadth, depth, and focus of assessment • Problem formulation: Technical implications of planning and scoping, with a conceptual model and an analysis plan • Early identification of decision-making options

12. Related concept – Value of information (VOI) • Decision makers face the tension between acting now or delaying decisions while research is conducted • VOI analysis offers a framework for systematic examination of this issue • Part of committee’s charge, topic of interest to EPA • Key point of emphasis: • VOI analysis is not possible in the absence of a structured analysis that includes information about risk management options and detailed uncertainty characterization

13. Conclusions about VOI • VOI analysis is technically challenging for any realistic problem structure, may place unrealistic demands on science • Needs knowledge of decision rules, well-defined uncertainty characterization, assumes primacy of risk assessment outputs • Formal VOI may be impractical in most settings, but the essential reasoning behind VOI can be adopted • Understanding the causal link between a specific source of information, how a decision-maker would change behaviors given this information, and how this could improve decisions • Similar concepts can be applied to consider the value of methods rather than the value of information

14. Uncertainty • A huge, cross-cutting topic… • There have been substantial differences among EPA’sapproaches to and guidance for addressing uncertainty in exposure and dose-response assessment. • The level of detail for characterizing uncertainty isappropriate only to the extent that it is needed to inform specific risk-management decisions appropriately. • Inconsistency in the treatment of uncertainty among components of a risk assessment can make the communication of uncertainty difficult and sometimes misleading. • Example: How do you interpret Monte Carlo analysis with emissions assumed to be “known”, exposure model with limited parametric uncertainty, and epidemiology with uncertainty only related to study’s statistical power?

15. “Tiers” of uncertainty analysisExample: WHO, 2007 • Tier 0: Default assumptions, single value for result • Tier 1: Qualitative but systematic identification and characterization of uncertainties • Tier 2: Quantitative evaluation of uncertainty making use of bounding values, interval analysis, sensitivity analyses • Tier 3: Probabilistic assessments with single or multiple outcome distributions reflecting uncertainty and variability

16. Variability • Variability in human susceptibility has not received sufficient or consistent attention in many EPA health risk assessments • Committee encourages EPA to move toward the long-term goal of quantifying population variability more explicitly in exposure assessment and dose-response relationships.

17. General conclusions re uncertainty and variability • EPA should encourage risk assessments to characterize and communicate uncertainty and variability in all key computational steps. • Uncertainty and variability analysis should be planned and managed to reflect the needs for comparative evaluation of risk management options. • In the short term, EPA should adopt a “tiered” approach for selecting the level of detail in uncertainty and variability assessments • This should be made explicit in the planning stage. • EPA should develop guidance on the appropriate level of detail needed in uncertainty and variability analyses • Provide clear definitions and methods for identifying and addressing different sources of uncertainty and variability.

18. Related topic: Defaults • Also called “inference options”, “default options”, “science policy”, “risk assessment policy” • The best choice for parameters/models on the basis of risk assessment policy in the absence of data to the contrary • By definition, cannot be proven correct/incorrect, but often has some scientific underpinning • First formalized in the Red Book

19. Selection and use of defaults • Established defaults need to be maintained for risk assessment steps that require inferences • EPA, for the most part, has not yet published clear, general guidance on what level of evidence is needed tojustify use of agent-specific data instead of a default. • Clear criteria should be available for judging whether, in specific cases, data are adequate for direct use or to support an inference in place of a default. • There are a number of defaults (missing or implicit defaults) that are engrained in EPA risk-assessment practice but are absent from its risk-assessment guidelines. • EPA does not quantify uncertainty when default assumptions are used.

20. Recommendations re defaults • EPA should • continue and expand use of the best, most current science to support and revise default assumptions. • develop clear, general standards for the level of evidence needed to justify the use of alternative assumptions in place of defaults. • work toward the development of explicitly stated defaults to take the place of implicit defaults.

21. Evidentiary standard for replacing default • “The committee recommends that EPA adopt an alternative assumption in place of a default when it determines that the alternative is ‘clearly superior’, that its plausibility clearly exceeds the plausibility of the default.”

22. Showing uncertainty when using defaults • To the extent feasible, EPA should move beyond qualitative description of uncertainty when default assumptions are used • Long term – improved probabilistic description of uncertainty commensurate with risk management needs • Short term – criteria for listing alternative values • Goal – Provide sensitivity analysis to illustrate impact of alternative assumptions and hence characterize robustness of risk estimates • Limit attention to assumptions with plausibility comparable to the default • Goal is not an exhaustive presentation of plausible estimates

23. Unification approach to dose-response assessment • Historically, dose-response assessments at EPA conducted differently for cancer and noncancer effects • Methods have been criticized for not providing the most useful results. • A consistent approach to risk assessment for cancer and noncancer effects is scientifically feasible and needs to be implemented.

24. Current approach

25. Thoughts re current approach • EPA has taken important steps to harmonize cancer and noncancer approaches, but with many scientific and operational limitations: • Noncancer effects do not necessarily have threshold or low-dose nonlinearity • The mode of action of carcinogens varies. • Background exposures and underlying disease processes contribute to population background risk, which can lead to linearity at the population doses of concern. • RfDs and RfCs do not quantify risk for different magnitudes of exposure but rather provide a bright line with limited use in risk-management decision-making • Cancer risk assessments usually do not account for human differences in cancer susceptibility (other than possible differences in early-life).

26. Dose-response relationship is dependent on heterogeneity in background exposure (endogenous and xenobiotic), biological susceptibility

27. Unification recommendations • A consistent, unified approach for dose-response modeling that includes formal, systematic assessment of • background disease processes and exposures • possible vulnerable populations • modes of action that may affect a chemical’s dose-response in humans. • Redefine the RfD or RfC as a risk-specific dose • provides information on the percentage of the population expected above or below a defined acceptable risk (with specific degree of confidence). • Formal introduction of variability into cancer dose-response modeling • Will require implementation and development • As new chemicals are assessed or old chemicals are reassessed • Of test cases to demonstrate proof of concept.

29. Diagnostic questions to aid dose-response assessment • What is known or suspected to be the chemical’s MOA? • What underlying degenerative or disease processes might the toxicant effect? • What are the background incidences and population distributions of these processes? • Are there identified sensitive populations? • What environmental contaminants in air, drinking water, food or in consumer products (e.g., in cosmetics) or endogenous chemicals (e.g., natural hormones) are similar to the chemical? • Could they potentially operate by MOAs similar to that of the chemical in question? • What chemicals might operate by a different MOA but have the potential to affect the same toxic process as the chemical under study? • Can subgroups with particularly high exposures be identified? • …more

30. Conceptual dose-response models – Based on MOA, background exposure and disease processes

31. General structures of the three models • Model 1: Estimate BMD, determine human POD with uncertainty, extrapolate linearly • Like current cancer framework, but applied to non-cancer • Model 2: Estimate BMD, determine human POD with uncertainty, determine risk-specific RfD given human heterogeneity • Like proposals in the literature by Hattis, Evans • Model 3: Estimate BMD, incorporate interindividual variability factor, extrapolate linearly while retaining variability • Like current cancer framework with variability factor

32. Cumulative risk assessment • EPA is increasingly asked to address broad public-health and environmental-health issues that stakeholder groups often consider inadequately captured by current risk assessments • multiple exposures • complex mixtures • vulnerability of exposed populations • There is a need for cumulative risk assessments as defined by EPA that include • combined risks posed by exposure to multiple agents or stressors • aggregate exposure to a given agent or stressor • all routes, pathways, and sources of exposure • Chemical, biologic, radiologic, physical, and psychologic stressors are considered.

33. Conclusions on cumulative risk • Committee applauds the agency’s move toward the broader definition, making risk assessment more informative and relevant to decisions and stakeholders. • However, in practice, EPA risk assessments often fall short of what is possible and supported by agency guidelines. • Little consideration of nonchemical stressors, vulnerability, and background risk factors. • Because of the complexity of considering so many factors simultaneously, there is a need for: • Simplified risk assessment tools • Orientation around pertinent risk management options to limit the number of stressors under formal consideration

35. Recommendations • Draw on other approaches to incorporate interactions between chemical and non-chemical stressors in assessments, including those from ecologic risk assessment and social epidemiology • Develop guidelines and methods for simpler analytical tools • to support cumulative risk assessment • to provide for greater involvement of stakeholders. • In short-term, develop databases and default approaches to allow for incorporation of key non-chemical stressors in the absence of population-specific data, considering • exposure patterns • contributions to relevant background processes • interactions with chemical stressors. • In long-term, invest in research programs related to interactions between chemical and non-chemical stressors, including epidemiologic investigations and physiologically-based pharmacokinetic modeling.

36. Improving the utility of risk assessment • Committee proposes a framework for risk-based decision-making • At its core are the “4 steps” as defined in the Red Book • Key difference from the Red Book in the initial and final steps • Framework asks implicitly: • What options are there to reduce the hazards or exposures that have been identified, and • How can risk assessment be used to evaluate the merits of the various options? • Risk assessment as a means to an end

37. Improving the utility (II) • Under this framework, the questions posed arise from • early and careful planning of the types of assessments (including risks, costs, and technical feasibility) and • the required level of scientific depth needed to evaluate the relative merits of the options being considered. • Risk management involves choosing among the options after the appropriate assessments have been undertaken and evaluated.

39. Phase I: Problem Formulation and Scoping • What is the problem to be investigated, and what is its source? • What are the possible opportunities for managing risks associated with the problem? Has a full array of possible options been considered, including legislative requirements? • What types of risk assessments and other technical and cost assessments are necessary to evaluate existing conditions and how the various risk-management options alter the conditions? • What impacts other than health and ecosystem threats will be considered? • How can the assessments be used to support decisions? • What is the required timeframe for completion of assessments? • What resources are needed to undertake the assessments?

40. Phase II:Planning and conduct of risk assessment Stage 1: Planning • For the given decision-context, what are the attributes of assessments necessary to characterize risks of existing conditions and the effects on risk of proposed options? • What level of uncertainty and variability analysis is appropriate? Stage 2: Risk Assessment Stage 3: Confirmation of Utility • Does the assessment have the attributes called for in planning? • Does the assessment provide sufficient information to discriminate among risk-management options? • Has the assessment been satisfactorily peer reviewed?

41. Phase III:Risk management • What are the relevant health or environmental benefits of the proposed risk-management options? • How are other decision-making factors (technologies, costs) affected by the proposed options? • What is the decision, and its justification, in light of benefits, costs, and uncertainties in each? • How should the decision be communicated? • Is it necessary to evaluate the effectiveness of the decision? If so, how should this be done?

42. The framework… • Systematically identifies problems and options that risk assessors should evaluate at the earliest stages of decision-making. • Expands the array of impacts assessed beyond individual effects (e.g., respiratory effects) to include broader questions (e.g., health status and ecosystem protection). • Provides a formal process for stakeholder involvement. • Increases understanding of the strengths and limitations of risk assessment by decision-makers at all levels, e.g., by making uncertainties and choices more transparent. • Maintains the conceptual distinction between risk assessment and risk management articulated in the Red Book.

43. Framework recommendation • EPA should adopt a framework for risk-based decision-making that embeds the Red Book risk assessment paradigm into a process with • initial problem formulation and scoping, • upfront identification of risk-management options, and • use of risk assessment to discriminate among these options.

44. Stakeholder involvement • Many stakeholders believe that the current process for developing and applying risk assessments lacks credibility and transparency. • Greater stakeholder involvement is necessary to ensure that the process is transparent and to ensure risk-based decision-making proceeds effectively, efficiently, and credibly. • Stakeholder involvement needs to be an integral part of the risk-based decision-making framework. • It is important that EPA adhere to its own guidance on stakeholder involvement particularly in the context of cumulative risk assessment, in which communities often have not been adequately involved.

45. Stakeholder recommendations • EPA should establish a formal process for stakeholder involvement in the framework for risk-based decision-making with: • time limits to ensure that decision-making schedules are met • incentives to allow for balanced participation of stakeholders including impacted communities and less advantaged stakeholders.

46. Concluding thoughts • The committee felt that, in spite of limitations, risk assessment remains essential to EPA’s mission to ensure protection of public health and the environment. • The committee hopes that the recommendations and the proposed framework for risk-based decision-making will provide a template for the future of risk assessment in EPA and strengthen the scientific basis, credibility, and effectiveness of future risk-management decisions.