2007 National Air Quality Conferences 13 February 2007, Orlando, FL Eric N. Schreffler, ESTC

1. Measuring the Emissions Reduction Impact of Episodic Public Education Campaigns and Issues Related to Year-round PM Programs 2007 National Air Quality Conferences 13 February 2007, Orlando, FL Eric N. Schreffler, ESTC

2. Purpose • Assist programs in San Francisco, Phoenix and New York • National review of ozone action program evaluations • Assess programs that evaluate travel and emission impacts • Focus on survey methodologies • Speculate about shift to year-round programs

3. Why Evaluate? • To document effectiveness of CMAQ funded programs • To provide management information on program effectiveness • To measure program progress and effectiveness over time • To provide information on impacts to the public and the media • To assess the cost effectiveness of public education programs in comparison to other mobile source emission reduction strategies • To quantify emission reductions as part of Ozone Early Action Compacts or Voluntary Mobile Source Emission Reduction Program (VMEP) credits in a SIP

4. Background • Developed quantification method for CARB/USEPA • Implemented and refined method in San Francisco, San Joaquin, and Chattanooga • Consulted to Phoenix evaluation effort • Bay Area interested in how others evaluate • New York developing new ongoing program

5. Evaluation Team • Eric N. Schreffler, Transportation Consultant (ESTC) • True North Research • EarthMatter Environmental Consulting • Arbor Planning and Management

6. Cities Included in the Research • San Francisco Bay Area • Phoenix • Austin • Louisville • Sacramento • Birmingham • Cincinnati

7. Bay Area Spare the Air Program • BAAQMD began summer “Spare the Air” (STA) program in 1991 • In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts • Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last

8. Bay Area Spare the Air Program • Definition of a driving reducer is someone who:1) purposely reduces trips on STA days, 2) could recall that it was an STA day, and 3) said they reduced trips for air quality reasons • Two critical data items from survey are : 1) proportion of reducers among drivers, and 2) average # of trips reduced • ARB/EPA methods recommends discounting self- reported number of trips reduced to account for over-reporting

9. Bay Area Program Evaluation Critical survey question is: Sometimes people will purposely decrease the amount of driving they do in a day. There are several ways people can decrease their driving, so let me ask you about each. Today, did you ride a bike for a trip that you normally would make by driving yourself? (If yes) How many trips did you reduce in this way?

10. Bay Area Program Evaluation Critical Findings: • 3-9% of drivers report reducing two trips in response to the program • three-quarters linking or eliminating trips and only one quarter changing driving mode

11. Bay Area Program Evaluation Means of Reducing Trips (2005):

12. Bay Area Program Evaluation TRAVEL IMPACTS 2005 2004 2003 2002 # of Drivers 4.73 million 4.84 million 4.75 million 4.75 million % Reducers 7.3% 7.2% 2.8% 3.6% Total Reducers 345,299 348,244 133,000 171,000 Avg. # Trips Red. 2.0 2.32 2.08 2.60 Adj. Trips Red. 1.0 1.16 1.04 1.46 Trip Reduced 345,299 403,963 138,054 249,600 VMT Reduced 2,173,176 2,595,705 959,584 1,610,708

13. Bay Area Program Evaluation EMISSION IMPACTS 2005 2004 2003 2002 TRAVEL IMPACTS Daily NOx Reduced 1.525 tons 2.023 tons 0.833 tons 1.776 tons Daily ROG Reduced 1.483 tons 1.945 tons 0.772 tons 1.86 tons Daily PM10 Reduced 0.525 tons 0.627 tons 0.23 tons 0.4 tons CONSUMER PRODUCTS IMPACTS Daily ROG Reduced 0.18 tons 0.23 tons 0.2 tons 0.24 tons

14. Cities that Measure Travel Behavior Impacts PHOENIX • Clean Air Campaign begun in 1987, includes summer ozone program • Conduct “pre- and “post-season” survey to gauge awareness and behavior • Survey asked “what, if anything, did you do when you heard the advisory?” • Last summer, returned to surveying on evening of alerts • Predominant behavior change – stay indoors more

15. Cities that Measure Travel Behavior Impacts AUSTIN • Clean Program a partnership, the Clean Air Force of Central Texas, begun in 1993 • Program now part of Ozone Early Action Compact • Add question to triennial regional MPO survey on knowledge of & reaction to program • Predominant behavior change – delay refueling and lawn moving

16. Cities that Measure Travel Behavior Impacts LOUISVILLE • Kentuckiana Air Education (KAIRE) Program; formerly Ozone Prevention Program • The program now targets both ozone and particulate matter • Surveys conducted in 2002 and 2004 as “pre- and post-season” surveys • Key indicator was likeliness, not actual travel behavior change • Most likely behavior change reported as trip linking or trip chaining

17. Cities that Measure Travel Behavior and Emission Impacts SACRAMENTO • Spare the Air campaign and annual evaluation survey begun in 1995. • Methodology involves surveying on alert days and on similar “control” days • Method is similar to ARB/EPA method used in Bay Area • Proportion of reducers is lower and average trips reduced higher than in Bay Area

18. Cities that Measure Travel Behavior and Emission Impacts BIRMINGHAM • Ozone Action Program begun in 1997: evaluations since 2003 • Surveys conducted on evening of alerts; three conducted in 2005 • Key question is “Do you take any actions in response to…alerts?” • Predominant behavior change - stay indoors more • Calculate emission reduction based in new carpools formed

19. Cities that Measure Travel Behavior and Emission Impacts CINCINNATI • Do Your Fair Share for Cleaner Air begun in 1994 • Evaluate original based on surveys and transit ridership counts • Questions added to regional survey used in 2002 • Predominant behavior change - delay refueling or lawn moving • Emission reduction based on %who say they reduce trips, assuming a round trip

21. COMPARATIVE FINDINGS • All the regions studied use RDD telephone surveys among adult residents within the program area. • Most of the targeted surveys ranged from 300-1,000; the multi- purpose surveys had larger samples. • Four of the programs survey on alert days; • two survey in the fall after the season; and • two use a before/after design to measure awareness and behavior right before and right after the summer ozone season.

22. COMPARATIVE FINDINGS • Most surveys measure travel behavior change by determining whether the respondent was aware of the program or the alert and then if they took any action in response. • Most program that measure travel behavior, but not emissions, assess the proportion of respondents who report making certain desirable changes in response to the air quality alert. • However, the proportion of reducers and the average number of trips reduced is crucial to estimating emission reduction impacts; some programs assume one round trip (not empirically based). • All four programs that currently measure emission reduction use average trip length from planning sources, rather than from air quality surveys, to estimate VMT reduction.

23. COMPARATIVE IMPACTS • Difficult to compare due to difference in approach, methods, survey questions and analytic techniques • Program experience relatively high awareness (51% - 90%) • Between 1%-12% of residents change travel behavior • Staying indoors, eliminating trip and linking trips most common means of reducing travel • Non-work (discretionary travel) trips are reduced more than commute trips

24. CONCLUSIONS • While most regions are not required to, many evaluate the travel and emission impacts of their OAD program and the methods are improving over time • Several medium-sized cities are including OAD programs in their Ozone Early Action Compacts • Most programs are not included in the region’s control strategy • Several program are becoming year-round “air quality” programs to account for PM and the 8-hour standard

25. IMPLICATIONS OF YEAR-ROUND • Ongoing just that…messages and education year-round • Staying indoors and postponing trips infeasible on an on-going basis • Mode shift may play a larger role • Cannot evaluate on episodic basis; impacts recall • Cannot evaluate on specific waves of marketing • Developing new methodology for New York, will call particulate alerts

26. METHODOLOGICAL IMPLICATIONS • Could track those who commit via website… • But that will miss indirect influences and is self-selective • Need “on-going” survey to assess incidence of preferred behavior and tease out reasons for behavior and influence of program or campaign • Issue: recall of program versus message • Issue: primary influence versus secondary • Would like to assess differential impact of ongoing versus episodic response and impacts

27. RECOMMENDATIONS • All involved air districts/agencies should continue to publish the annual survey results and program impacts and broadly disseminate the information. •   An informal working group could be formed among agencies that operate or evaluate public education programs and could convene at the annual National Air Quality Conference to compare experiences and coordinate. • National guidance on quantification of on-going program impacts could be developed. •   Survey and evaluation results could be posted on the AIRNOW website (www.airnow.gov) and enable internet discussions of methods and findings.

28. THANKS! Eric N. Schreffler Transportation Consultant estc@san.rr.com CARB/USEPA methodology available at: www.arb.ca.gov/research/abstracts/98-318.htm