Building Community Collaboration to Promote Healthy Youth Development:

1. Building Community Collaboration to Promote Healthy Youth Development: Social Development Research Group School of Social Work University of Washington www.sdrg.org J. David Hawkins Ph.D. Melissa Institute May 1, 2008

2. The Challenge for Community Prevention • To address those risk factors most prevalent in a community with tested, effective policies and programs. • To address protective factors most depressed in a community with tested, effective policies and programs that strengthen those protective factors.

3. The Communities That Care Prevention System • Helps communities apply the advances of prevention science to guide youth development and prevention work. • Measures community levels of protection and risk by surveying young people. • Matches the community’s profile of risk and protection with tested, effective programs and policies.

4. The Communities That CarePrevention System • Local control builds ownership to create sustainable change. • Focuses on outcomes to insure success: Are fewer teens using drugs? Fewer smoking? Fewer committing violent acts?

5. The Communities That Care Operating System Get Started Creating Communities That Care Implement and Evaluate Get Organized Develop a Profile Create a Plan

6. The Communities That Care Operating System Get Started Creating Communities That Care Implement and Evaluate Get Organized Develop a Profile Create a Plan • Community readiness assessment. • Identification of key individuals, stakeholders, and organizations.

7. The Communities That Care Operating System Get Started Creating Communities That Care Implement and Evaluate Get Organized Develop a Profile Create a Plan • Training key leaders and board in CTC • Building the community coalition.

8. The Communities That Care Operating System Get Started Creating Communities That Care Implement and Evaluate Get Organized Develop a Profile Create a Plan • Collect risk/protective factor and outcome data. • Collect information on community resources • Construct a community profile from the data.

9. Peer-Individual Estimated National Value

10. The Communities That Care Operating System Get Started Creating Communities That Care Implement and Evaluate Get Organized Develop a Profile Create a Plan • Define outcomes. • Prioritize factors to be targeted. • Select tested, effective interventions. • Create action plan. • Develop evaluation plan.

11. Factor Addressed Program Strategy Developmental Period Family Management Problems Prenatal/Infancy Programs prenatal-2 Early Childhood Education 3-5 Parent Training prenatal-14 Family Therapy 6-14 Addressing Barriers with Effective Action

12. Effective Training for Middle School Parents • Guiding Good Choices (Spoth et al., 1998) • Adolescent Transitions Program (Dishion and Andrews, 1995) • Parenting Adolescents Wisely (Gordon et al., 1998) • Creating Lasting Connections (Johnson et al., 1996) • Strengthening Families 10 to 14 Program (Spoth, 1998) • Focus on Families (Catalano et al., 1999; 1997)

13. The Communities That Care Operating System Get Started Creating Communities That Care Implement and Evaluate Get Organized Develop a Profile Create a Plan • Form task forces. • Identify and train implementers. • Sustain collaborative relationships. • Evaluate processes and outcomes. • Adjust programming.

14. What is required to install CTC? • A coalition of community stakeholders. • A coordinator for the CTC process. • Manuals and curriculum materials. • Training from certified trainers. • Technical assistance when difficulties are encountered. • A monitoring system to provide routine feedback on progress and outcomes.

15. CTC Trainings • Key Leader Orientation • Community Board Training • Community Assessment Training • Community Resource Assessment Training • Community Planning Training • Community Program Implementation Training

16. Communities that CareProcess and Timeline Measurable Outcomes Process Increase in priority protective factors Decrease in priority risk factors Increase in positive youth development Reduction in problem behaviors Assess risk, protection and resources Implement and evaluate tested prevention strategies Vision for a healthy community 6-9 mos. 1 year 2-5 years 5-10 years

17. The Community Youth Development Study (CYDS) • A 24 community randomized controlled trial to test the Communities That Care system started in 2003.

18. The Community Youth Development Study Team Abigail A. Fagan John Graham Kevin Haggerty Koren Hanson J. David Hawkins David M. Murray Sabrina Oesterle M. Lee Van Horn Robert D. Abbott Michael W. Arthur Megan M. BaldwinJohn S. Briney Blair Brooke-Weiss Eric C. Brown Rick Cady Richard F. Catalano Funded by: National Institute on Drug Abuse Center for Substance Abuse Prevention National Cancer Institute National Institute on Child Health and Development National Institute on Mental Health

19. CYDS State Collaborators • Colorado Alcohol & Drug Abuse Division • Illinois Division of Community Health & Prevention • Kansas Alcohol and Drug Abuse Services • Maine Office of Substance Abuse • Oregon Office of Alcohol & Drug Abuse Programs • Utah Division of Substance Use • Washington Division of Alcohol & Substance Abuse

20. CYDS Primary Aim To test the efficacy of the Communities That Care system in • reducing levels of risk • increasing levels of protection • reducing health and behavior problems among adolescents using a true experimental design.

21. STUDY DESIGN Randomized Controlled Trial 2003-2008 2003 2004 2005 2006 2007 2008 Implement selected interventions Planning 5-Year Baseline 1997-2002 CTCYS CTCYS CTCYS CKI CRD CKI CRD Intervention 98 99 ‘00 ‘01 ‘02 CTC Board CTC Board CTC Board CTC Board CTC Board Randomize CTCYS CTCYS CTCYS YDS YDS YDS YDS YDS CKI CRD CKI CRD CTCYS CTCYS CTCYS Control CKI CRD CKI CRD CTCYS: Cross-sectional student survey of 6th-, 8th-, 10th-, and 12th-grade students using the CTC Youth Survey CKI: Community Key Informant Interview CRD: Community Resource Documentation measuring effective prevention programs and policies in the community  CTC Board: CTC Board Member Interview YDS: Longitudinal Youth Development Survey of students in the class of 2011 starting in 5th grade in spring 2004 YDS YDS YDS YDS YDS

22. Demographics of 24 CYDS Communities

23. Panel-Youth Development Survey (YDS) • Annual survey of panel recruited from the Class of 2011 (5th grade in 2004) • Active, written parental consent

24. Youth Development Survey • Participants recruited in grades 5 and 6. • Final consent rate = 76.4%

25. 2006 YDS • 96.2% Overall Student Participation • 10.3% (n=454) had moved out of project schools

26. 2007 YDS • 96.2% Overall Student Participation • 11.9% (n=525) have moved out of project schools

27. Communities That CareLogic Model CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes

28. Adoption of Science-Based Prevention • Stage 0: No Awareness • Stage 1: Awareness of Prevention Science Terms and Concepts • Stage 2: Using Risk and Protection Focused Prevention Approach as a Planning Strategy. • Stage 3: Incorporation of Community Epidemiological Data on Risk and Protection in Prevention System. • Stage 4: Selection and Use of Tested and Effective Preventive Interventions to Address Prioritized Risk and Protective Factors. • Stage 5: Collection and Feedback of Process and Outcome Data and Adjustment of Preventive Interventions Based on Data. Note. Community Key Informant Survey (CKI).

29. Baseline Stages of Adoption by Intervention Status Control Communities CTC Communities Probability Stage of Adoption in 2001 Note. Community Key Informant Survey (CKI); N = 534; γ001 = .037, SE = .395, df = 20,p > .05 .

30. Post-Intervention Stages of Adoption by Intervention Status Control Communities CTC Communities Probability Stage of Adoption in 2004 Note. Community Key Informant Survey (CKI); N = 534; γ101 = -1.311, SE = .355, df = 20, p < .002 .

31. Collaboration onPrevention • Prevention Collaboration • Assessed by 9 items measuring prevention-specific collaborative activities. There is a network of people concerned about prevention issues who stay in touch with each other. Organizations in [community] share money or personnel when addressing prevention issues. Organizations in [community] participate in joint planning and decision making about prevention issues. 1=strongly agree, 2=somewhat agree, 3=somewhat disagree, 4=strongly disagree Note. Community Key Informant Survey (CKI).

32. Control Communities CTC Communities Change in Prevention Collaboration by Intervention Status Factor Score Post-Int (2004) Pre-Int (2001) Year Note. Community Key Informant Survey (CKI); N = 530; γ001 = -0.123, SE = .118, df = 20, p > .05; γ101 = 0.237, SE = .094, df = 20, p < .022.

33. Communities That CareTheory of Change CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes

34. Program Selection CTC Community Board members selected prevention programs from a menu* of programs that: • Showed significant effects on risk/protective factors, and drug use, delinquency, or violence • Involved at least one high-quality research study • Targeted children or families in grades 5-9 • Provided materials and training * Communities That Care Prevention Strategies Guide

35. Programs Selected in 2004-2007 *Program funded through local resources in one or two communities

36. Exposure in the Community Note: Total eligible population of 6th, 7th, and 8th-grade students in 2005-06 was 10,031. *Includes PALS, BBBS, Stay SMART, and Tutoring programs

37. Participant Attendance Percentage attending >60% of the total number of sessions *Includes PALS, BBBS, Stay SMART, and Tutoring programs

38. Fidelity Assessment Checklists • Obtained from developers (9) or created by research staff (7) • Provided similar information across all programs to measure 4 elements of fidelity • Over 6,000 checklists were completed by program implementers and coordinators • Minimal missing data (8.2% in 2004-05 and 2.1% in 2005-06) • Checklists were collected and reviewed by communities, then sent to SDRG

39. Adherence Rates2004-05 and 2005-06 school years Percentage of material taught or core components achieved

40. Adherence • Average adherence rate were high: 91% in 2004-05 and 94% in 2005-06 • “Problematic” modifications were infrequent (1.2 reported per cycle) • Deletions reported twice as often as additions • Implementer reports of challenges: • Lack of time (14-20% of checklists) • Participant misbehavior (9-11%) • Trouble engaging participants (4-9%)

41. Delivery of Lessons2004-05 and 2005-06 school years(number, length, and frequency of required sessions) Percentage of delivery requirements met

42. Delivery of Lessons • Delivery scores were high: 91% and 92% in 2004-05 and 2005-06 • “Problematic” deviations in delivery requirements were infrequent • Deviations most likely to occur in school-based programs • e.g., 30 vs. 45-minute sessions • e.g., deleting 1 of 12 required sessions

43. Program Observations • Observed 10-15% of sessions in 10 of 16 programs • Completed fidelity checklists to verify adherence information • Rate of agreement between observers and implementers was 93% (range: 77%-100%) • Observers also rated the quality of delivery and participant responsiveness

44. Quality of Delivery • Observers rated the quality of delivery on 10 items (alpha = .83-.88) using a 5-point scale (higher scores indicate better quality) • Example Items: • In general, how clear were the program implementer’s explanations of activities? • To what extent did the implementer keep on time during the session and activities? • Rate the implementer on the following qualities: • Level of enthusiasm • Rapport and communication with participants • Effectively addressed questions/concerns

45. Quality of Delivery 2004-05 and 2005-06 school years Average score on 10 items reported by program observers

46. Quality of Delivery Diversity of Teaching Techniques • Observers rated the percentage of the session in which each teaching technique was used

47. Participant Responsiveness • Observers rated participant responsiveness on two items, using a 1-5 scale (higher scores indicate better responsiveness): • To what extent did the participants appear to understand the material? • How actively did group members participate in discussions and activities? • Across all programs, rates were high: 4.38 and 4.52 in 2004-05 and 2005-06

48. Communities That CareTheory of Change CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes

49. Prioritized Risk Factors in CTC Communities • Family management problems • Parental attitudes favorable to problem behavior • Family conflict • Low commitment to school • Favorable attitudes toward problem behavior • Friends who engage in problem behavior • Academic failure • Rebelliousness • Laws and norms favorable toward drug and alcohol use

50. Pre-post Change inTargeted Risk Factors Average Level of Risk Grade 5 Grade 7 Note. Values are model-fitted levels of standardized average risk for students in the Youth Developmental Study panel sample, N = 4404. For Grade 7: γ010 = .111, SE = .043, df = 11, p < .026.