Measuring the Impact of Environmental Strategies in Prevention

1. Measuring the Impact of Environmental Strategies in Prevention Jeremy T. Goldbach, PhD, LMSW C.A.R.E. Consulting Group This product is supported by Florida Department of Children and Families Office of Substance Abuse and Mental Health funding.

2. Learning Objectives After participation in this workshop, participants will be able to : • Explain the advantages of environmental strategies • Use data to identify environmental strategies that may be a good fit in their own communities • Measure the impact of environmental strategies in prevention

17. What’s the point?!

18. So Why Environmental Strategies? • A significant portion of youth ages 12 to 17 in the United States have used alcohol and other drugs. • Rates of binge drinking (five or more drinks) in underage youth are a concern, with large increases between 7th grade (1.6% of 12 and 13 year olds) and high school graduation (34.7% of 18 to 20 year olds). • Economically, the costs of alcohol use and its related consequences in U.S. communities add up to more than $235 billion annually (Rehm, et al, 2009). (NSDUH, 2009; Rehm et al. 2009)

19. Why environmental strategies Continued • Addressing alcohol abuse through only one strategy (such as a single direct service prevention intervention only) is less effective than taking a broader approach that includes the larger community. • multi-component strategies promote a systemic approach to solving issues recognizing that individuals operate within, and are affected by, an “environment”. (Hawley, 1950; Rousseau & House, 1994; Stokols, 1996).

20. Breaking it Down What are Environmental Strategies? Making flowers bloom? Ending Wars? Getting rid of pollution? NO…well, not for us anyway!

21. What Are Environmental Strategies? Effect is on a population Environmental strategiesdo nothaveto be complex

22. Environmental Strategies Environmental strategies focus on the following areas: • Access & availability • Policy/laws and their enforcement • Community norms/mediamessages

23. Access & Availability Are the amount of time, energy, and money that must be expended to obtain a substance (alcohol, tobacco, drugs). The more resources it takes to get the substance (harder), the less the availability (access).

24. Policies/Regulations/Laws They control availability, support norms, and specify sanctions for violations.

25. Norms The rightness orwrongness, the acceptability or unacceptability, and/ordeviance of specific behaviors for a specificgroup of individuals.

26. Why Environmental Strategies? • Broad reach – universal population • Substantial effects • Immediate and enduring effects • Cost effectiveness • Ease of maintenance

27. Share Your Experience • Discuss successes you’ve had in using environmental strategies • Discuss barriers/struggles in using environmental strategies

28. Selecting the Right Environmental Strategy • How do you know which type of environmental strategy is best for your state or community? • What do you need to know to decide which is best? • Can you do just one or should you do more than just one?

29. Profile population needs, resources, and readiness to address needs and gaps Sustainability & Cultural Competence Monitor, evaluate, sustain, and improve or replace those that fail Mobilize and/or build capacity to address needs Implement evidence-based prevention programs and activities Develop a Comprehensive Strategic Plan The SPF Framework Assessment Evaluation Capacity Implementation Planning

30. Why Use the SPF? • Effective capacity building process • Required by SAMHSA and used by many others • Gives a roadmap based on research, data, and evidence • Decisions can be evaluated through data-driven measurement

31. Step 1: Needs Assessments • Why do we assess? • To guide the prevention effort • To provide information about the target population and their corresponding substance abuse problem

32. Substance Abuse and Related Consequences Risk and Protective Factors Evidence- Based Prevention Strategies Key Elements

33. Types of Risk/Protective Factors: “Why?” • Retail access • Social access • Pricing • Norms around Alcohol Issues • Enforcement • Promotion

34. Risk Factors for Binge Drinking Substance Abuse & Related Consequences Risk/Protective Factor Strategies Binge Drinking Social Access/ Availability Retail Access

35. Risk Factor Assessment: “But Why Here?” • When does this happen? • Where does this happen? • How does this happen? • Who is affected? • Who allows this to happen?

36. But WhyHere? Substance Abuse & Related Consequences Risk/Protective Factor Strategies Binge Drinking Social Access/ Availability: Parents in community host underage drinking parties

37. Step 2: Capacity Building • To find out what we already have and what we need • Different types of capacity (fiscal, physical, staff, etc) - So we don’t duplicate efforts or pick much-needed, but unachievable strategies

38. Step 3: Planning • What kind of planning is done? • Identifying the most powerful factors (intervening variables) contributing to the target behavior • Investigating appropriate evidence-based strategies to match population’s needs • Planning expected outcomes (both short and long-term)

39. Types of Environmental Strategies • Policy • Enforcement • Communication

40. Selection of Evidence-Based Strategies Relevant? Demonstrates “Conceptual Fit” Appropriate? Demonstrates “Practical Fit” Effective? Demonstrates “Evidence of Effectiveness” Best Fit Prevention Intervention

41. Evidence-Based Environmental Strategies Risk/Protective Factor Strategies Substance Abuse & Related Consequences Binge Drinking Social Access Enforcement Social Host Ordinances Communication Social Marketing “Parents Who Host Lose the Most”

42. Key Elements for Success of Environmental Strategies • Use data for decision-making • Fit:relevant, appropriate and effective • Good mix of strategies • Monitor implementation and outcomes

43. Step 4: ImplementING • What do we do? • Develop an action plan for implementing the strategic plan • Take fidelity of implementation into account • Write detailed evaluation plan that includes process and outcome measurements

44. Step 5: EvaluatING • What do we evaluate? • The effectiveness of the selected evidence-based program, policy, or practice (EBP) • How? • By collecting required outcome data • Recommendations made for quality improvement

45. What is an Evaluation? “The most serious mistakes are not being made as a result of the wrong answers… The truly dangerous thing is asking the wrong question.” – Peter Drucker • Asks questions to understand Who, What, When, Where, Why, and How things are being done • Identifies strengths and barriers to the coalition process • Identifies strengths in the implementation of your strategies • Identifies gaps and areas for improvement • Tells you about your PROCESS as well as your OUTCOMES

46. Evaluation is Important Because It…(I promise, there are real answers!) “One of the great mistakes is to judge policies and programs by their intentions rather than their results” – Milton Friedman • Helps to improve program planning • Monitors the effectiveness of strategies • Advances knowledge in the prevention sciences • Serves as a mechanism to build capacity by involving community in evaluation • Can encourage the community to publicize success stories from the prevention effort

47. Process Evaluation • What is process evaluation? EXAMPLES: • What is it like to be a new member of the coalition? • To what extent is our strategy being implemented as planned? • What obstacles were encountered during implementation?

48. Outcome Evaluation Over the duration of the program, to what extent has: • Coalition membership increased? • Community-wide prevention awareness activities changed adult norms about substance use? • Youth alcohol use decreased?

49. How Do We Explore These Questions Effectively? • Each activity that the coalition does should have a purpose • Each activity should be measureable and link logically to an outcome of interest

50. The Logic Model