Under Pressure to Evaluate your HHW Program? Evaluation Concepts in a Tire Waste Program

1. Under Pressure to Evaluate your HHW Program?Evaluation Concepts ina Tire Waste Program Jennifer J. Tabanico California State University

2. Program Evaluation • The process of collecting, analyzing, and interpreting information about our intervention efforts that are aimed at some stated goal • Need • Implementation • Effectiveness • Efficiency

3. Why We Don’t Evaluate • Not required • Complicated statistics • Lack of knowledge about research methods • Extra Cost • “We’re doing something”

4. Why Evaluate? • Required by funding agencies • Justify program existence • Preparing for budget cuts • Can make changes • Unintended consequences • Cost efficiency • Inform future efforts

5. Types of Evaluations • Summative Evaluation • Program goals • Did it work? • Formative Evaluation • Program process • Why worked/didn’t work

6. Setting Measurable Goals • Identify specific goals • What do you want to achieve? • Specific behavior change • Public satisfaction • Waste reduction • Identify your measures • How will you know if you reached your goals? • Surveys • Observations • Waste

7. Stages of Evaluation • Needs assessment • What is happening, and what should be happening? • Program planning • Theory based development of intervention • Implementation and Process • Is program being implemented correctly? • Outcome analysis • Is program causing the desired change? • Is program worth the cost?

8. The Tire Waste Problem • 31 million waste tires per year generated by California motorists (CIWMB, 2003) • Proper maintenance increases life of tire and reduces waste • Pressure, Alignment/Balancing, Rotation, Tread

9. Overview of Intervention • Goal • Increase frequency and accuracy of tire pressure checking behavior by motorists • Program • Motorists at 10 gas stations • 5 experimental, 5 control • Motivational messages + free pressure gauge at each pump at 5 stations • Outcomes • Survey of 381 motorists (self report & tire inspection) • Air pressure machine usage

11. Needs AssessmentBaseline Survey Data • “Checking pressure” is the most under performed maintenance behavior • Only 27 % checked pressure monthly as recommended • 59% of vehicles on the roadways had at least one tire over/under inflated by 5+ psi • Motorists know they should check • Don’t know correct PSI and ~50% don’t have gauge

12. Program PlanningApplying Theoretical Principles • Give Aways • Norm of reciprocation • Social Norms • Beliefs about what other people think/do • Motivation • Survey data revealed saving money, extending tire life, and safety as motivations to act

13. Implementation and ProcessEvaluation of Implementation • 7,565 tire gauges distributed at the pump at 5 gas stations • Dispensers refilled steadily for 4 weeks • Penetration Rates • Self report of gauge ownership at experimental vs. control stations

14. Do you own a tire pressure gauge?

15. Outcome AnalysisDid it work? • Experimental vs. Matched Controls • Follow up survey of motorists • Self reported behavior • Tire inspections • Air pressure machine usage • Unobtrusive electronic device

16. Self-Reported Behavior

17. Tire Inspections

18. Air Pump Usage

19. Conclusions • Evaluation provides information about need, implementation, effectiveness, and efficiency of program • Is it working? • What is/isn’t working? • Can apply these principles to own program • Workshops, staff training on evaluation principles a good investment

20. References • California Integrated Waste Management Board. (2003a). Waste tire management program:2001 staff report. May 2003, Publication # 620-03-003 http://www.ciwmb.ca.gov/Publications/default.asp?pubid=1006. • California Integrated Waste Management Board (2003b). Consumers’ tire buying habits and their knowledge of tire maintenance, recycling, and disposal. November 2003,Publication # 622-03-004. • Cialdini, R. B. (2000). Influence: Science and practice (4th ed.). Boston: Allyn & Bacon. • National Highway Transportation Safety Administration. (2002). Tire pressure survey and test results. http://www.nhtsa.dot.gov/cars/rules/rulings/TirePressure • Rubber Manufacturers Association. (2002). Be tire smart: Play your PART. http://www.rma.org/tiresafety/tiremaintenanceandsafety. • Schultz, P. W. (2004). Community Based Social Market Pilot to Increase Proper Tire Maintenance. Final report submitted to the California Intergrated Management Board. • Schultz, P. W., & Oskamp, S. (2000). Social psychology: An applied perspective. Upper Saddle River, NJ: Prentice-Hall.