Enhancing Program Evaluation for Federal Environmental Research

1. Session 853 Extending Organizational Capacity & Capability to Evaluate Federal Environmental Research ProgramsResearch Contributions to Outcomes & Accountability for the Acid Rain Program David Schmeltz* & Rona Birnbaum EPA Office of Atmospheric Programs October 29th, 2005 *Corresponding author contact:schmeltz.david@epa.gov 2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association

2. Overview • Issues of concern • Framework for accountability • A new accountability driver

3. Primary Issues of Concern • Sulfur, nitrogen and mercury deposition • Atmospheric processes and transport • Total deposition • Multiple effects over multiple scale • Acidification • Eutrophication • Nitrogen saturation • Ozone impacts • Mercury contamination

4. Ecological Areas Sensitive to Atmospheric Deposition

5. Sulfate Deposition and Acid-Sensitive Surface Waters Wet Sulfate Deposition to Acidic Surface Waters 2004 The greatest sulfur and nitrogen deposition occurs in areas of the Midwest and northeastern United States which are downwind of the highest SO2 and NOx emission areas. Impacts occur in both the eastern U.S. and mountainous areas of the West. Despite substantial emissions reductions over the last 20 years, high levels of sulfur and nitrogen deposition still enter acid-sensitive lakes and streams, leading to high levels of acidity.

6. Nitrogen Deposition and Forested Ecosystems • Under current emissions rates, nitrogen saturation is expected to get worse • Nitrogen deposition is a significant problem in many western areas, including the Colorado Front Range, the San Gabriel Mountains, the Klamath Mountains, and the San Bernadino Mountains • This is leading to high nitrogen levels in streams in several areas and changing the ecological structure of some alpine lakes and tundras • Nitrogen saturation contributes to greater forest and grassland susceptibility to fire Points on map represent only those forested areas surveyed for these purposes

7. Coastal Ecosystems: Eutrophication is a regional scale problem NOx Airsheds for Four Estuaries Hudson/Raritan, Chesapeake, Pamlico, and Altamaha • Understanding what air programs can do to protect coastal ecosystems requires a regional, interstate perspective

8. Program Accountability Drivers • What is the effectiveness of regulation/policies? • Are we meeting the goals to improve/protect ecosystems? • Performance measures • GPRA (Government Performance and Results Act) • PART (Program Assessment Rating Tool) • Reporting requirements due to statute and international agreement (e.g. NAPAP Report to Congress, US-Canada Air Quality Agreement Progress Report, etc.) • Self-imposed reporting requirements (e.g. Acid Rain Program Progress Report, NOx Budget Program Progress Reports, etc.) • A new driver: National Academy of Sciences 2004 Report Recommendations on Air Quality Management • It’s the right thing to do!

9. Focus on the Acid Rain Program Goals: GPRA In the past, performance measures set forth under GPRA were used to assess progress under the Acid Rain Program • Tons of SO2 and NOx emissions reduced • Total annual average sulfur deposition and mean ambient sulfate concentrations reduced (% from baseline) • Total annual average nitrogen deposition and mean ambient nitrate concentrations reduced (% from baseline)

10. Focus on the Acid Rain Program Goals: PART Using more robust outcome-based performance measures to assess program effectiveness set forth under PART • SO2 emissions reduced (tons/yr) • Total annual average sulfur and nitrogen deposition and mean ambient sulfate and nitrate concentrations reduced (% from baseline) • Number of chronically acidic water bodies reduced (% from baseline year level) • Working towards developing a health-based outcome measure

11. Program Theory: Outcomes and Key Research Contributions Intermediate Environmental Outcomes Problem Definition – Nature and Extent of Environmental Problem; Effectiveness & Cost Scientific Knowledge & Tools to Measure Multiple Effects at Multiple Scales Environmental Measurements Intermediate Environmental Outcomes Short-Term Outcomes Long-Term Outcomes Decisions about: Emissions - cap levels and timing Changes in: EmissionsAtmospheric concentrations Deposition Visibility Changes to: Sensitive receptors in aquatic and forested ecosystems Improvements in:Ecosystem health Human health Scientific Knowledge & Tools that Link Emissions to Air Quality & Deposition Scientific Knowledge & Tools that Link Changes in Sensitive Receptors to Ecosystem Health Scientific Knowledge & Tools that Link Air Quality & Deposition to Sensitive Receptors

12. Acid Rain Program: State-by-State SO2 Emissions Levels 1990 to 2004

13. Monitored Reductions in Wet Sulfur Deposition in the Eastern U.S. Wet Sulfate Deposition Average 1989-1991 Wet Sulfate Deposition Average 2002-2004 • Acid Rain Program SO2 goal: Reduce SO2 emissions from electric generators by 8.5 million tons (50% below 1980 levels) • In 2004, SO2 emissions from all power generation were 10.3 million tons, 5.4 million tons (34%) below 1990 levels • Eastern states have experienced significant decreases in sulfate deposition – about 30 percent -- since the Acid Rain Program took effect in 1995

14. What are the Impacts of Deposition Changes on Ecosystems: Long-term Surface Water Monitoring Trends -- Chemistry TIME/LTM (Surface Water Monitoring) Regional Trends in Lakes and Streams Acidity, 1990-2000 • Regional declines in surface water sulfate can be directly linked to declines in emissions and deposition of sulfur • In three regions, one-quarter to one-third of lakes and streams previously affected by acid rain are no longer acidic • Regional Acid Neutralizing Capacity (ANC), a key indicator of recovery, did not change significantly in New England or in Blue Ridge streams • Surface water nitrate concentrations are largely unchanged except in Adirondacks and Northern Appalachian Plateau

15. Regions Where Recovery Has Begun 14 Before Title IV 12 Current 10 8 % lakes chronically acidic 6 4 2 0 Adirondacks Upper Midwest Northern Appalachian Plateau Lakes and Streams are Beginning to Recover from Acidification • However, there are others where it has not: • 5.5% of New England lakes remain chronically acidic, an insignificant change from the early 1990s when 5.6% of lakes in the region were acidic • There has also been no change in the number of acidic waters in the Blue Ridge region of Virginia in the past decade

16. ORD Research Contributions: TIME/LTM Monitoring Network • Extremely crucial source of information on ecosystem response and progress in reducing number of chronically acidic water bodies • Data and findings (i.e., Stoddard report) used in other program reporting (e.g., NAPAP Report to Congress, Acid Rain Program Progress Report, etc.) • Multipollutant policy development • Critical to understanding why ecosystems are or are not responding to emission controls (i.e., ecosystem processes)

17. Environmental Monitoring Networks • Long-term • Environmental Scale (over space and time) • Commitments • Network map represents a history of collaboration, years of relationships in designing, developing and maintaining networks • Networks mirror scale and nature of assessment needs • Networks serve as platforms for broad range of research uses and users

18. Where do we go from here? • Assessment is not an end in and of itself; it is the input for revisiting and improving policy decisions • Further emissions reductions • Further monitoring and assessment • Further program development (e.g. emissions cap and trade, standard setting, etc.) • The recent National Academy of Sciences Report on Air Quality Management charges EPA to institutionalize accountability

19. Emerging Programmatic Needs for Ecological Research • Developing tools to assess changes in ecosystem condition in relation to new air quality regulations or legislation • New ecological indicators/benchmarks • Linkages between deposition and effects • Vulnerability mapping and assessment • Multi-pollutant accountability (e.g., routine mercury deposition monitoring methods)

20. NAS Report on Air Quality Management • January 2004 -- NAS Committee on Air Quality Management in United States releases report • Comprehensive assessment of effectiveness of US air quality management system • Core conclusions: • Over past 30 years, Clean Air Act has substantially reduced pollution emissions • Despite progress, Committee identified scientific and technical limitations that will hinder future progress • Report intended as blueprint to address limitations, enhance air quality management, and chart path toward more productive and efficient system • Viewed as opportunity for EPA and to “step outside the box” to achieve better environmental results