Economic Value of Improved Risk Assessment Data in Environmental Regulation

1. A Decision Analytic Framework for Considering the Economic Value of Improved Risk Assessment Data Eric Ruder Henry Roman Industrial Economics ARA Workshop IV - May 23, 2012

2. IEc Overview of Presentation • Objective: Explore the potential value of improved risk assessment data and methods in the context of environmental regulation • Nature of the problem • Analytical framework • Examples –Regulatory Case Studies • Lead Regulation • Mobile source related air toxics requirements • Implications for ARA Framework 1

3. Nature of the Problem – Evaluating Improvements in Risk Assessment IEc • There are numerous sources of uncertainty in risk assessment that may be addressed (at least partially) by improved risk assessment methods and/or data: • Improving methods for evaluating the potential risks of chemicals – e.g., mode of action or other dose-response assessment methods. • Identifying susceptible populations and characterizing factors that may place them at higher risk. • Understanding how chemicals move and change along pathways from sources to potentially exposed populations. • Advances in these areas seem of value, but are they likely to lead tangible changes in regulatory decisions that have quantifiable net benefits for society? 2

4. IEc Value of Information Analysis • Value of information (VOI) represents the improvement in the expected value of a decision outcome that would result from collecting additional information about one or more factors affecting a decision. • Decision analysis approaches can quantify the value of collecting additional information before making a specific decision, or the value of adopting improved methods of data analysis. • Involves comparison of the expected outcomes of a suite of alternative regulatory choices made with and without information believed critical for the issue. 3

5. IEc Why is VOI Important? • Can help decision-makers make better decisions • identifies key decision inputs • helps minimize expected loss / maximize expected gain • increases chances of good outcome • makes explicit the "costs" of uncertainty • Results provide a measure of social willingness-to- pay for new research • Helps set priorities so that resources can be allocated efficiently 4

6. IEc Evaluations of VOI as a Tool • NRC’s Science and Decisions: Advancing Risk Assessment (2009) • Discusses VOI as a tool for understanding the tradeoffs between timely decision-making and the desire to refine the underlying science • Barriers to applying formal VOI broadly • Underlying concepts and structure still valuable: linkages between information, decision-maker behavior, and decision making objectives • Recommends informal VOI or value of methods analyses to inform risk assessment design 5

7. IEc Evaluations of VOI as a Tool (cont’d) • 2010 Report of EPA’s Board of Scientific Counselors (BOSC) decision analysis workshop with ORD/NRMRL  VOI can be valuable, but is challenging to implement o Need extensive data, including probabilities for decision options & how probabilities change conditional on new information  Nonetheless, VOI cited as possible method to inform several case studies – e.g., regulations of chemicals with biomarker data, prioritizing IRIS evaluations  Benefits of VOI are the net improvements in decision outcomes minus the costs of obtaining improved information 6

8. IEc VOI Analytic Framework • Decision analysis approaches can quantify the value of collecting additional information before making a specific decision • Involves comparison of the expected outcomes of a suite of alternative regulatory choices made with and without information believed critical for the issue • Framework in a regulatory context can be illustrated using two types of diagrams:  Influence diagram - showing the role of toxicity data and other inputs in estimating NSB  Decision tree – can depict a choice among (for example): o Taking no further regulatory action o Instituting additional control measures based on current information o Collecting additional dose-response information prior to deciding whether to institute new controls 7

9. IEc VOI Framework Influence Diagram Dose- Response Information Effectiveness of Controls Value of Health Effects Change in Health Effects Change in Exposure Monetized Benefits (Avoided Health Effects) Baseline Exposure Implement Controls? Control Costs Net Benefits 8

10. IEc VOI Framework (continued) Hypothetical Decision Tree: VOI = $500 - $437.5 = $62.5 Toxicity Control Decision Policy Decision Net Social Benefits No Controls $0 MOA A .25 $1,000 [$437.5] Controls MOA B .5 $500 MOA B .25 $-250 No Controls $0 MOA A .25 Controls $1,000 Collect Exposure Information No Controls $0 [$500] MOA B .5 Controls $500 No Controls $0 MOA C .25 9 Controls All values in millions of $ $-250

11. IEc VOI Framework (continued) • Variations in complexity of analysis: • Value of perfect information -- assumes that collection of information will eliminate all uncertainty in exposure • Value of imperfect information -- examines the potential that addition of information will resolve some, but not all of the uncertainty in exposure • Value of partial information -- considers the impact of uncertainty in other inputs to the decision (e.g., for the previous hypothetical example - toxicity) 10

12. IEc Lead Case Study - Background Decision we analyzed (retrospectively): regulating lead in residences under TSCA Section 403 Rule sets three standards: floor dust loading (µg/ft2); window sill dust loading (µg/ft2); and soil concentration (ppm). Violation triggers abatement Economic and risk analysis supporting the rule looked at 1000 combinations of these three standards Risk and economic estimates ultimately based on two alternative models for relationship between environmental lead and children’s blood lead:  IEUBK: more sensitive, says regulate stringently  The “Empirical” model: much less sensitive, says do not regulate November 1998 proposed standards reflect EPA’s balancing of these two exposure outcomes • • • • • 11

13. IEc VOI Case Study - Lead Exposure COMPARISON OF NET BENEFITS, DUST STANDARDS, AND SOIL STANDARDS FOR TWO LEAD UPTAKE MODELS Standards that Maximize Net Benefits IEUBK Model Proposed Standards 40 g/ft2 100 g/ft2 250 ppm $100.4 billion $273.6 billion $173.2 billion Floor Dust Standard Window Sill Dust Standard Soil Standard Total Cost Total Benefit Net Benefit 50 g/ft2 250 g/ft2 2,000 ppm $52.8 billion $160.1 billion $107.2 billion Standards that Maximize Net Benefits Empirical Model 80 g/ft2 310 g/ft2 4,350 ppm $44.0 billion $35.1 billion -$8.9 billion Floor Dust Standard Window Sill Dust Standard Soil Standard Total Cost Total Benefit Net Benefit 12

14. IEc Lead Rule RIA Influence Diagram LEAD RULE REGULATORY IMPACT ANALYSIS INFLUENCE DIAGRAM Lead Dose- Response Function Lead Abatement Effectiveness (Adjustment Factors) Baseline Exposure (HUD Survey Data) Change in Exposure (Blood Lead Distribution) Change in Persons with IQ<70 Change in IQ (points/person) Lead Uptake Efficiency Value of IQ Changes Implement Abatement Actions? Monetized Benefits (Avoided IQ Changes) Net Benefits Abatement Costs 13

15. IEc Decision Tree for Lead Regulatory Decision VOI = $43.3b - $19b = $24.3b No_Controls [0] 0 IEUBK .250 Empir .750 IEUBK .250 Empir .750 IEUBK .250 Empir .750 $173.2 HighControls [$17.1] $-34.9 Medium_Controls [$19] $107.2 $-10.4 $77.9 Low_Controls [$12.8] $ -8.9 High_Controls $173.2 Medium_Controls $107.2 IEUBK .250 [$173.2] Low_Controls $77.9 No_Controls Exposure_Data [$43.3] $0 High_Controls $-34.9 Medium_Controls $-10.4 Empir .750 [$0] Low_Controls $-8.9 No_Controls 14 $0

16. Effect of Delay on VOI IEc • Delay scenarios assume: • Benefits are delayed if “collect exposure data” option is chosen. • Discounted over delay period at 3 percent annually. • Results: • Perfect VOI, no delay: $24.3 billion • Perfect VOI, five-year delay: $18.4 billion • Perfect VOI, ten-year delay: $13.2 billion. 15

17. IEc Value of Imperfect Information • Value of imperfect information analysis asks: What if improved data cannot completely resolve uptake uncertainty? • Looked at best and worst case scenarios for power of NHEXAS-like data to resolve IEUBK/Empirical uncertainty (from 50 percent predictive accuracy to 90 percent predictive accuracy) • Results show VOI remains substantial when improved data achieves a high level of predictive accuracy and the prior likelihood that IEUBK is correct is less than 50 percent 16

18. Expected Value of Imperfect Lead Exposure Information IEc 18 16 14 Initial Likelihood IEUBK is Better Model 12 0.1 0.2 EVI (Billions of $) 0.3 10 0.4 0.5 0.6 8 0.7 0.8 0.9 6 4 2 0 0.5 0.6 0.7 0.8 0.9 Predictive Pow er of Improved Exposure Data 17

19. IEc Conclusions of Lead Case Study • Value of exposure information could be very high; under best case conditions, a national exposure survey could have value in the tens of billions of dollars. • Effect of delay in provision of information is to reduce VOI, but value of perfect information remains substantial even with 10-year delay. • Improved exposure information must achieve a predictive accuracy of 80 to 90 percent certainty in forecasting the "correct" lead uptake model for VOI to remain substantial (e.g., no more than a 20 percent chance of predicting IEUBK if the empirical model is correct); lower levels of certainty quickly erode VOI. 18

20. IEc VOI Case Study: Mobile Air Toxics • Retrospective analysis of VOI of exposure data for benzene and 1,3-butadiene, two air toxics regulated under the 1990 CAAA. • Developed estimates of exposure and risk in 2000 both with and without control programs -- based on information available as rules were promulgated. • Other inputs: control costs (over $2 billion annually); other benefits of air toxics reductions (about $850 million annually). • Estimate value of improved information for mobile source air toxics relative to improvements in other key uncertain inputs (e.g., toxicity). 19

21. Mobile Source Air Toxics - Exposure Uncertainty IEc Alternative Exposure/Distributions for 1,3-Butadiene 30% 25% Percent of Population 20% 5th 25th 50th 75th 95th 15% 10% 5% 0% 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Concentration (ug/m3) 20

22. IEc Calculation of Net Social Benefits CALCULATION OF NET SOCIAL BENEFITS Exhibit 2 Exposure Characterization Other Benefits and Control Costs Estimation of Direct Health Benefits Health Effects Valuation Potency Other Health and Welfare Benefits of Control Measures Baseline (No-Control) Exposure Distribution1 (g/m3) Baseline Health Damages Baseline Risk Net Social Benefits of Emission Controls Direct Health Benefits Health Effects Valuation Potency Post-Control Exposure Distribution1 (g/m3) Post-Control Health Damages Cost of Control Measures Baseline Risk 21 1 This includes characterizing the uncertainty in the exposure description.

23. IEc Decision Tree for Motor Vehicle Air Toxics No_Control [0]* Control [$103] But_exp [$141] N5th .15 But_exp [$160] N25th .225 But_exp [$187] N50th .25 But_exp [$220] N75th .225 Decision_1 No_Control Decision2 [0] $0 N5th .15 Collect_Exposure_Data Control Benz_exp [$196] -$240 No_Control Decision2 [0] $0 N25th .225 Control -$13 No_Control But_exp [$281] Decision2 [$190] $0 N95th .15 N50th .25 Control $190 No_Control Decision2 [$435] $0 N75th .225 Control $435 No_Control Decision2 [$901] VOI = $93 million (in 1995 $) $0 N95th .15 Control $901 22 * Numbers in brackets are expected values of net social benefits at decision or chance nodes.

24. Conclusions of Mobile Sources Case Study IEc • Expected value of perfect information = $93 million. • Explored value of partial information by considering impact of uncertainty in toxicity of 1,3-butadiene and benzene.  Value of exposure information decreases to $9 million.  Value of toxicity information is greater, $118 million. • Explored impact of varying dollar value of a statistical life (VSL).  Exposure information has value across a wide range of VSL (approximately $3 million to $7 million). 23

25. IEc Applications / Requirements • VOI framework can be applied to variety of issues (pollutant regulation, global warming) and focus on different inputs (exposure, toxicity, economic valuation) • A key constraint for formal quantitative VOI is data availability. Need data to: • estimate outcomes (ideally, net social benefits) in dollars • characterize quantitatively decision inputs (e.g., costs, population exposure, toxicity) and how they change post- information • quantify uncertainty either discretely or using distributions pre- and post-information -- may require expert elicitation 24

26. IEc Identifying Decisions Where VOI May be High • High Stakes, sufficient variance in outcomes across choices • Decision expected to be sensitive to health outcomes and not dictated by non health-related factors • Uncertainty can be represented using small number of scenarios (e.g., alternative MOAs) and associated probabilities • Additional research (e.g., improved PBPK modeling, interpretation of biomarker data, MOA research) can change beliefs about likelihoods of key decision factor values/states • Uncertainty in other decision inputs does not dominate uncertainty in input for which information is collected 25

27. VOI and the ARA Framework IEc • Potential approaches for a case study in the context of the ARA Framework  Full quantitative VOI o Can be challenging and resource intensive o Best reserved for quantitative, in-depth assessments for decisions with substantial impacts  Informal, qualitative VOI (as discussed in Science & Decisions) o Must establish causal link between research/methods and reduced uncertainty in toxicity AND link between reduced uncertainty and decision-maker’s choice o Explore how to operationalize this for qualitative assessments or quantitative screening assessments  Are there existing case studies that can serve as foundation for a VOI analysis 26

28. IEc INDUSTRIAL ECONOMICS, INCORPORATED Eric Ruder, Principal eruder@indecon.com Henry Roman, Principal hroman@indecon.com\ 617.354.0074