Anne E. Smith, Ph.D. Vice President asmith@crai (202) 662-3872

1. Air Quality Health Risk Assessment –Methodological Issues and Needs Presented to SAMSISeptember 19, 2007Research Triangle Park, NC Anne E. Smith, Ph.D. Vice President asmith@crai.com (202) 662-3872

2. Risk Assessment for Ambient Air Pollutants • A policy analysis primarily performed by environmental protection agencies to assess benefits of air quality regulations • The analysis step where clinical and epidemiological evidence of “whether” there is a health effect shifts to asking “how much” is the public health being affected. • Focus of this presentation is on issues in performing quantitative portion of an air quality risk assessment • Important not to lose sight of the qualitative elements of RA • Typical quantitative risk assessment questions: • How much does ambient pollution affect public health outcomes? • How much would public health improve if we changed ambient air quality standards?

3. Ambient Air Environmental Risk Assessment • Part of formal USEPA air quality standards-setting process • Frequently in the headlines

4. Analysis of the added uncertainties is paramount…and a substantial challenge for analysts Quantitative Risk Assessment Requires Extrapolations • From epi associations  “dose-response” function • From observed current ambient air  changes in “dose” MultipleEpi-derived Ests of Toxicity What is true toxicity value? What is true shape of f(.)? Policy“Benefits” # Health Events = f( ΔExposure* Toxicity) Current AmbientConcentrationDistributions Changes dueto Policy How will policy actuallyaffect concentration distribution?

5. Epidemiological associations are quantitative, but that does not mean they should be interpreted at face value as “exposure-response functions”. Discussion of Uncertainties in the “Dose-Response” Function Portion of a Risk Assessment

6. Uncertainties in Using Epidemiological Associations for Extrapolating to Public Health Outcomes • Are the associations evidence of a causal relationship? • Causality with respect to the specific ambient pollutant that the air quality standard applies to is especially uncertain • How should uncertainty on causality enter into risk estimates? • Assuming causality: • Impact of exposure error on quantitative estimate of toxicity • Impact of unknown missing explanatory variables • Toxicity estimates from “multi-pollutant” vs. “single-pollutant” models • Unknown “correct” lag structure • Very limited capability of epidemiological estimates to identify any non-linearities in underlying concentration-response function • Particularly important for extrapolation to much lower ambient concentration levels than studied

7. 3 Cohorts Studied “6 Cities” “Am Cancer Institute” “Veteran’s Admin” “2-pollutant” formulation “1-pollutant” formulation “1-pollutant” formulation “1-pollutant” formulation “2-pollutant” formulation Single estimate used in USEPA’s deterministic risk assessment Example: Multiple Estimates in Epidemiological Literature on Chronic Exposure Mortality Risk from Fine Particulate Matter (PM2.5) For details of analysis: Anne Smith, Comments to USEPA, 3/31/07 , (σ) <0 (signif) <0 (signif) <0 (signif) <0 (insignif) , (σ) <0 (signif) , (σ) .005 (.001) .006 (.002) .006 (.003) .006 (.004) .004 (.002) .002 (.001) , (σ) .003 (.002) .012 (.004) .004 (.004) .002 (.004) , (σ) .013 (.004) .008 (.004)

8. USEPA RA’s 95% confidence intervalbased on single regression Example of Results of Integrated Uncertainty Analysis for PM2.5 Mortality, Compared to Deterministic Results Estimates of percent of long-term mortality incidence attributed to PM2.5 -- Los Angeles for 2003 ambient levels – NoPM2.5 risk at all 6-8% 4-6% 0-2% 2-4% 8-10% 10-12% 12-14% 14-16% 16-18% 18-20% 20-22% 22-24% 24-26% = 3684 deaths per year in Los Angelesdue to 2003 PM2.5 concentrations For documentation of Integrated Uncertainty Analysis assumptions, see “Appendix C” of Anne Smith’s Comments on the Second Draft of EPA’s PM2.5 Risk Analysis, 3/31/05 (available in PMDocket or on request from author). EPA RA estimates are from Final Risk Assessment (July 2005)

9. Illustrativeassignments of weights for uncertainty analysis .33 .33 .33 .5 .5 .5 .5 “1-pollutant” formulation “2-pollutant” formulation Single estimate used in USEPA’s deterministic risk assessment Example: Multiple Estimates in Epidemiological Literature on Chronic Exposure Mortality Risk from Fine Particulate Matter (PM2.5) For details of analysis: Anne Smith, Comments to USEPA, 3/31/07 3 Cohorts Studied “6 Cities” “Am Cancer Institute” “Veteran’s Admin” “2-pollutant” formulation “1-pollutant” formulation “1-pollutant” formulation , (σ) <0 (signif) <0 (signif) <0 (signif) <0 (insignif) , (σ) <0 (signif) , (σ) .005 (.001) .006 (.002) .006 (.003) .006 (.004) .004 (.002) .002 (.001) , (σ) .003 (.002) .012 (.004) .004 (.004) .002 (.004) , (σ) .013 (.004) .008 (.004)

10. 60% USEPA RA’s 95% confidence intervalbased on single regression 50% 40% 30% 20% 10% 0% Histogram of pdf from Illustrative Integrated Uncertainty Analysis Example of Results of Integrated Uncertainty Analysis for PM2.5 Mortality, Compared to Deterministic Results Estimates of percent of long-term mortality incidence attributed to PM2.5 -- Los Angeles for 2003 ambient levels – NoPM2.5 risk at all 6-8% 4-6% 0-2% 2-4% 8-10% 10-12% 12-14% 14-16% 16-18% 18-20% 20-22% 22-24% 24-26% For documentation of Integrated Uncertainty Analysis assumptions, see “Appendix C” of Anne Smith’s Comments on the Second Draft of EPA’s PM2.5 Risk Analysis, 3/31/05 (available in PMDocket or on request from author). EPA RA estimates are from Final Risk Assessment (July 2005)

11. Discussion of Uncertainties in the Exposure Portionof a Risk Assessment

12. Large share of change assumed to occur on days with relatively low pollutant concentration (e.g., well below the standard) Key RA assumption: how each part of the pollutant exposure distribution would be affected by a change in the ambient standard Uncertainties in Pollutant Exposure Changes due to Policy: The Pollutant “Rollback” Assumption -- Rollback Alternative 1 -- 100% 90% 80% 70% Cumulative Frequency 60% 50% 40% 30% 20% 10% 54 57 69 72 48 51 60 63 66 21 24 27 30 33 36 39 42 45 12 15 18 0 3 6 9 Daily 8-Hour Maxima (PPB)

13. Larger share of change assumed to occur on days with relatively high pollutant levels Rollback if there is a substantial “background” or uncontrollable portion of pollutant along the distribution Uncertainties in Pollutant Exposure Changes due to Policy: The Pollutant “Rollback” Assumption -- Rollback Alternative 2 -- 100% 90% 80% 70% Cumulative Frequency 60% 50% 40% 30% 20% 10% 54 57 69 72 48 51 60 63 66 21 24 27 30 33 36 39 42 45 12 15 18 0 3 6 9 Daily 8-Hour Maxima (PPB)

14. Needs for Methodological Development

15. Bottom Line of these Examples & Research Needs • Health risk assessment for air quality standards is subject to enormous uncertainty • Key uncertainties are not statistical • Model selection and model shape for exposure-response • Modeling of air quality distributional changes as result of policy • Initial examples of methods to incorporate the above forms of uncertainty into the standard risk assessments produce very different characterization of risk and of benefits from air quality standards • The challenge to the risk analysis community is in developing workable approaches for • Integrating multiple sources of uncertainty • Dealing with the highly judgmental aspects required to represent the key sources of these uncertainties

16. Coda: Estimated Annual “Lives Saved” from Revised PM2.5 Ambient Standard (as published with Final Rule) EPA’s Regulatory Impact Assessment, p. ES-8: Some use of uncertainty analysis as rule was finalized -- based on direct elicitation -- did not address any uncertainties in exposure (“rollbacks”) -- still many issues to be addressed …and other pollutants (e.g., ozone)

17. Disadvantages in the epi-based risk analysis situation: The vast majority of “experts” are those who wrote the epidemiological papers (motivational bias) By the time risk analysis is starting, most potential experts are viewed as having a “position” on the decisions that the risk analysis will inform Advantages in the epi-based risk analysis situation: Experts can be disinterested outsiders to the study of the pollutant in question; need to be experts only in their ability to interpret regression-based studies generally “Evidence-driven”: creates explicit linkage of weights to subjective opinions about relative quality of underlying studies Explicit criteria for “quality” in a model can be articulated It is possible to “blind” the experts to the quantitative estimates in each study (minimization of potential motivational bias) Two Approaches for Characterizing Probability Distributions on Concentration-Response Relationships Top-down method “Direct Elicitation”: Ask “experts” what they believe to be likelihood of different levels of pollutant’s risk, including their personal uncertainty. Bottom-up method, “Evidence-Driven Elicitation”: Ask experts to assign weights to all relevant epidemiological model results, based on their personal views about the relative quality of each model formulation/study design, etc.