Richard Wilson, D. Phil., MA Harvard University Special Lecture at the University of Kuwait

1. Value of Information (VOI) Analysis A Tool for Considering Research Priorities:Are fine particles bad for your health? Richard Wilson, D. Phil., MA Harvard University Special Lecture at the University of Kuwait 13th January 2003

2. Basic Framing of Issue • Decisions about environmental control are frequently complicated by uncertainty. • In many cases uncertainty could be reduced by further research. • Decision makers face the tension between acting now or delaying control decisions while research is conducted. • VOI analysis offers a framework for systematic examination of this issue.

4. Hypothetical Problem • In the absence of further control, current levels of exposure to a contaminant leads to population risk, R=1 (cases/yr). We assign $1 million to each case • There are three things which can be done • no control - no cost and no risk reduction • moderate control - 0.25 M $/yr and 50% risk reduction • stringent control - 0.81 M $/yr and 90% risk reduction • Decision criterion is maximization of net benefits (or minimization of total social costs).

5. Total Social Costs(with no uncertainty)

6. Total Social Costas a Function of Unknown Risk

7. Uncertain Estimate of Risk

8. Total Social Costwith Uncertainty in R

9. Opportunity Loss • The opportunity loss is the difference between … • the total social cost of the strategy that was chosen on the basis of the expected value of risk, and • the total social cost of the strategy that would have been chosen if the true value of risk had been known before the decision was made.

10. Expected Opportunity Loss • The EOL is the expected value of the Opportunity Loss, i.e.: • the integral of the product of the Opportunity Loss at each specific possible value of R and the probability density function for R. • The EOL is also called the Expected Value of Perfect Information (EVPI) and is a bound on the value of any actual information.

12. John Evans, Sc.D. John Graham, Ph.D. James Hammitt, Ph.D. T.J. Carrothers, Sc.D. Scott Wolff, Sc.D. Jonathon Levy, Sc.D. Jouni Tuomisto, MD, Ph.D. Andrew Wilson, S.M. Joshua Cohen, Ph.D. Example: Risks from PM Research funded by US Environmental Protection Agency ORD as an element of Harvard PM Research Center and also by unrestricted HCRA funds.

13. Fine Particle Emissions:Million Tons/Year (US 1990) Secondary Primary Secondary Total Source Category PM NO SO 2.5 x 2 Utility Coal-Fired Power Plants 0.1 6.7 15.2 22.0 Other 0.0 0.7 0.6 1.4 Total Utility 23.4 15.9 0.1 7.4 Motor Vehicle Light Duty Gasoline Vehicle 0.0 3.4 0.1 3.6 Heavy Duty Diesel Vehicle 0.2 2.3 0.3 2.9 Other 0.0 1.7 0.1 1.8 Total Motor Vehicle 0.3 0.6 8.3 7.4 Other Source Classes 2.8 8.1 0.6 4.6 Total Industrial Point 1.0 3.2 2.2 6.4 Total Area 0.3 2.8 Total Nonroad 0.2 3.4 Total Estimated US Air Emissions 4.5 22.7 22.4 49.6 1990 U.S. Air Emission Estimates by Source Category (million tons / year) Reference: US EPA 1997

14. Fine Particle Control Costs 40 35 30 25 20 Emissions Reduction (Million Tons / Yr) 15 10 5 0 0 5 10 15 20 25 30 35 40 45 Cost ($ Billion/Yr)

15. Key Questions • Which controls should be implemented? • What are the critical areas of scientific uncertainty? • How do these uncertainties affect the decision? • What are the potential benefits of research that could reduce these critical uncertainties? • Which research strategies have the greatest potential and promise?

16. Our Approach • Based on decision analysis and value of information analysis. • Decision criterion is maximization of expected net benefits. • Sequential • Preliminary Decision and VOI Analyses • Uncertainty Analysis/Expert Elicitation • Final Decision and VOI Analyses

17. Exposure Acute Mortality Efficiency Coefficient LYs per Acute Death LYs per Population Emissions Acute Deaths Chronic Exposure Death Health Costs $ per Life or Lifeyear Chronic Control Chronic Deaths Mortality Strategy Coefficient Total Social Control Costs Cost Fine Particulate Matter: Influence Diagram

18. Name of Control Source Particle Emissions Reduction Cost 6 9 Type (10 U.S. ton / yr) (10 / yr) Fabric Filter CFPP Primary 0.10 $ 0.95 Fuel Switching CFPP Sulfate 4.35 $ 0.82 2 Flue Gas Desulfurization CFPP Sulfate 1.96 $ 0.55 2 Low NOx Burner CFPP Nitrate 3.57 $ 8.30 x SCR/(Cyclone Boilers) CFPP Nitrate 0.38 $ 0.27 x Oxidation Catalysts Mobile Primary 0.05 $ 0.51 Low Sulfur Fuel Mobile Sulfate 0.36 $ 1.53 2 3 Way Catalysts Mobile Nitrate 3.55 $ 21.50 x Exhaust Gas Recirculation Mobile Nitrate 1.39 $ 5.30 x Cost and Effectivenessof Illustrative Control Strategies(Coal-Fired Power Plants & Mobile Sources) Notes: CFPP - Coal-Fired Power Plant; 1997 $ Source: Wolff (2000). Sc.D. Thesis, Harvard School of Public Health.

19. Fate and Transport • Focus on Power Plants and Mobile Sources • National perspective • Evaluated for 40 sites (randomly selected) • Estimated using CALPUFF • Meteorological data for 1990 (US EPA) • Sulfate & nitrate formation - Mesopuff rates • 448 receptors (100 x 100 km) • Model uncertainty (primary 50%; sulfate 2x, nitrate 3x) • Expressed as Exposure Efficiency

20. CFPP and Mobile Source Locations (Wolff, 2000) CFPP locations Mobile source locations

21. Range and Distribution of EE for Coal-Fired Power Plants • Lognormal with GM = 1.9x10-6 and GSD = 1.7 for Primary PM2.5 • Potentially useful for risk assessment by analogy, adaptation by few descriptor variables

22. Exposure Efficiency • Coal Combustion Emissions • Primary FP 2.0 x 10-6* / 1.5 • Secondary Sulfate 2.0 x 10-7* / 2 • Secondary Nitrate 1.0 x 10-7* / 3 • Mobile Source Emissions • Primary FP 9.0 x 10-6* / 1.5 • Secondary Sulfate 2.0 x 10-7* / 2 • Secondary Nitrate 1.0 x 10-7* / 3

24. Meta Analysis (Levy, JAWMA 2000)

25. Acute Mortality • Dose-response coefficient (PM2.5) • Central estimate 0.7% per 10 mg/m3 • Lower, upper estimates 0.1, 1.3% per 10 mg/m3 • Source: NMMAPS (2000) • Judgmental plausibility of effect • Probability observed effect is caused by fine particles = 90% • Length of life lost/death • Central estimate 2.5 QALY/d (Coronary Heart Failure) • Lower estimate 0.5 QALY/d (“Sickest of Sick”) • Upper estimates 6.5 QALY/d (80% CHD; 20% normal) • Source: Kuntz CVD model

26. Life Expectancy by Age:General Population vs. Persons with Coronary Heart Disease (CHD)

28. Survival Curve from Six Cities

29. Cohort Mortality - Six Cities Study (Dockery et al., 1993)

30. Chronic Mortality • Dose-response coefficient (PM2.5) • Upper estimate 13% per 10 mg/m3 (Six Cities) • Lower estimate 5% per 10 mg/m3 (ACS) • Source: (Dockery, 1993; Pope, 1995, Krewski, 2000) • Judgmental plausibility of effect • Probability effect is due to fine particles = 50% • Length of life lost/death • Approximately 10 discounted QALY/d • Calculated as difference between cohort and time series • Assumptions: 15 y/d; 5 y latency; 3% discount rate

33. Relative Toxicity of Various Particles • Base Case • Primary (C) = Secondary (S & N) • Primary (C) = 2 x Secondary (S & N) • Primary (C) = 1/2 x Secondary (S & N) • Sensitivity Analysis • Primary only • Sulfate only • Nitrate only • Note crustal assumed to be non toxic

34. NMMAPS Time-Series Results (Samet et al., 2000)

35. Source-Specific PM2.5 and Daily Mortality in Six US CitiesLaden, Neas, Dockery, Schwartz, EHP 2000 • PM2.5associated with daily mortality in six cities (1980’s) • Factor analysis of elemental composition of PM2.5used to estimate source-specific concentrations • Associations estimated with 4 source classes (10 mg/m3) • Crustal (Si) • Motor Vehicle (Pb) • Coal (Se) • Residual Oil (Vn, Mn)

36. Source-Specific PM2.5 and Cause-Specific Mortality in Six US Cities COPD & Pneum CVD

37. Valuation • Two basic approaches • Value Deaths (VSL) • Value Change in (Quality Adjusted) Life Expectancy (VSLY) • Basic analysis relies on VSL approach • Central value $6,000,000 • Low value $2,000,000 • High value $10,000,000 • With 10 discounted QALY/d these correspond to VSLY of $300K, $100K, and $500K, respectively

38. Baseline Results (VSL Case) 60 50 40 30 Expected Net Benefit (billion $/year) 20 10 0 SCR - C.F.P.P. Low NOx Burner...- CFPP Mobile Oxidation Fuel Switching - Catalysts - CFPP Low S Fuel - Mobile Mobile Flue Gas De-S 3way Cat... - EGR (HDDV) - CFPP - Mobile CFPP Fabric Filter -

39. Expected Value of Perfect Information -- Base Case (VSL)

40. 10 5 0 -5 -10 Expected Total Net Benefits (billion $/year) -15 -20 -25 -30 Results without Chronic Effect (VSL)

41. 14 12 VOI Total 8 8 VOI - Believe Chronic 8 7 8 7 10 7 5 8 Expected VOI (Billion$ / y) 5 6 5 4 3 2 0 70 10 80 20 30 40 50 60 90 100 0 Prior Probability (Believe Chronic) Sensitivity Analysis: Prior Probability for “Believe Chronic” (VSL)

42. Implications • The potential benefits of research are quite large, • Research which could … is vital. • Influence the plausibility that the cohort studies reflect a causal relationship with PM • Reduce the uncertainty in estimates of population exposures to PM • More clearly indicate the relative toxicity of particles emitted from various sources • But good research may take time ...

43. The Costs of Waiting (VSL) 400 300 200 100 EVPI (billion $) 0 1 2 3 4 5 6 7 8 9 -100 -200 -300 Time (years) * Example computed using 3% social discount rate

45. Peer Review • Approach is Sound • Preliminary Work Provides Good Foundation • Major Strength Is Integrated Analysis • Suggestions • Borrow Exposure Results from EPA • Consider Differential Toxicity More Carefully • Consult with EPA on Control Strategies

46. Next Steps • Review and revision of model • Expert judgment workshops • Interpretation of Health Evidence (Cohort Studies and Relative Toxicity) • Evaluate Promise of Alternative Research Strategies • Recompute value of perfect and partial information.

47. Approach to Elicitation of Expert Judgment • Review Background Materials • Select Experts • Conduct Expert Workshop • Discuss Key Scientific Issues • Review Literature on Expert Judgment • Conduct Calibration Exercise • Structure Probability Tree • Elicit Individual Judgments • Analyze Results

48. Probability Tree

49. Summary • VOI has the potential to be helpful in setting priorities for applied research. • It requires one to formally represent the decision alternatives, their costs and effectiveness. • It requires one to quantitatively characterize the state of knowledge and uncertainty. • Quantitative results may not be as important as using the decision and voi model to carefully frame the discussion.

50. A Final Thought • He who knows and knows he knows • He is wise -- follow him • He who knows and knows not he knows • He is asleep -- wake him • He who knows not and knows he knows not • He is a child -- teach him • He who knows not and knows not he knows not • He is a fool -- shun him