Comprehensive Assessment: A Data Driven Decision Making Process

1. Comprehensive Assessment: A Data Driven Decision Making Process

2. University of South Carolina Student Life A caring community united for a safer Carolina. To create a campus-community environment that promotes healthy and safe behaviors among faculty, staff, students, and community organizations. VISION MISSION

3. University of South Carolina Student Life PURPOSE To build capacity and to perform ongoing assessment, planning, implementation, and evaluation of substance abuse prevention strategies, and to make recommendations for policy change GOAL To contribute to the overall reduction of high risk substance use and its consequences at the University of South Carolina

4. The SPF process enables campus communities to: Plan and develop substance abuse prevention infrastructure Implement effective prevention strategies Environmental approaches and educational initiatives Achieve population level outcomes A Model for Reducing High Risk Drinking Behaviors on Campus: The Strategic Prevention Framework ASSESSMENT Profile population needs, resources, & readiness to address needs and gaps EVALUATION Monitor, evaluate, sustain, and improve or replace those that fail CAPACITY Mobilize and/or build capacity to address needs Sustainability & Cultural Competency IMPLEMENTATION Implement evidence-based Prevention programs & activities PLANNING Develop a Comprehensive Strategic Plan

5. Our Coalition’s Assessment Process

6. Assessment Timeline 2014-15

7. Purpose Goal Objective Data, Assessment and Evaluation Subcommittee To inform and guide the coalition in problem and asset identification, strategic planning and implementation, and outcome evaluation PURPOSE To conduct ongoing, comprehensive community-campus assessment and process/outcome evaluation GOAL • To To conduct assessment to determine the extent of: • Risk factors associated with the problem • Protective factors (assets and resource availability) • Community-campus readiness to address the problem and fill needs and service gaps • To conduct process and outcome evaluation of prevention strategies OBJECTIVE

8. Data, Assessment and Evaluation Subcommittee Tasks of the Subcommittee • Determine the assessment’s purpose and scope • Define the assessment’s goals and objectives • Select the approach and methods for collecting data • Design and pilot test the collection instruments and procedures • Prepare assessment timelines • Collect the data • Analyze the data • Prepare data in a way that describes the root causes and local conditions that contribute to identified community problems • Develop a problem statement(s) that our coalition will address on a community wide, environmental level Source: The Fenway Institute

9. Comprehensive Assessment Three Areas: • Problem Identification and Prioritization • Resource Identification • Readiness for Change

10. Comprehensive Assessment: #1: Problem Identification and Prioritization The subcommittee has identified many indicators that help to explain student substance use and patterns. Specifically we look at: • prevalence of use • locations of use • drinking patterns • consumptions levels • student subpopulation(s) most affected • types of substances being used/abused • average age of use • environmental factors, such as availability, accessibility, and cultural norms

11. Comprehensive Assessment: #1: Problem Identification and Prioritization • Enterprise Risk Management • Risk Assessment (probability, severity, velocity) • Risk Control Assessment • Final Risk Score and Ranking

12. Comprehensive Assessment:#2: Resource Identification Identify past and current organizational and individual resources, and categorize those resources by key community sectors. Identify future resources needed to achieve our desired outcomes Ask the following questions: • Who has the knowledge we need? • Who has the time to assist? • Who has the human resources we need? • Who can hold us accountable for outcomes? • Who can provide financial support? • Who can encourage us and recognize our successes? • Who has helped with this issue in the past? • Who has an interest in our outcome? • Who has decision making power? • Who cares about this issue? • Who has other resources like space, food, giveaways, that we may need?

13. Comprehensive Assessment:#2: Resource Identification Identify quantitative and qualitative data currently being gathered and analyzed. • AlcoholEdu: Freshmen Student Survey • CORE: Upperclassmen Student Survey • NCHA • STIR: Indicated Student Population • OSC: Student violations and sanctions related to substance use • USCPD: crime data related to substance use • Health and Wellness Center: health consequences related to substance use • Key Informant Interviews • Focus Groups • Environmental Scanning

14. Comprehensive Assessment: #3: Readiness for Change Determine important aspects to consider when assessing readiness to change. Considerations include: • Examining multi-levels and multi-facets of campus and the surrounding community • Determining political will for change • Measuring shared resolve/commitment to implement change • Determining if there is a shared belief in our capability to make change • Determining if there is a shared value that change needs to occur • Assessing the campus community’s ability to meet task and resource demands • Concluded that the higher the readiness for change, the more likely change will be initiated, the greater the effort , persistence, and cooperation members will give

16. Drinking LocationFall 2013 vs. Fall 2014

18. Problem Identification and Prioritization 2014-15 Priority: Bars and nightclubs Priority: Off campus residences Priority: Other high risk substance use Priority: Community organizing and advocacy 1 2 3 4

19. Steering Committee utilized assessment information to: Identify and prioritize the problem(s) Develop logic models to identify root causes and contributing local factors Coalition members then formed subcommittees to: Create SMART Goals Select the intended outcome(s) and measure(s) Identify intermediate variables and measures Identify strategies for changing intermediate variables Conduct Strategic Prevention Planning

20. USC Problem Priority : Prevalent Drinking in Off Campus Residences Root Causes Contributing Local Factors Problem Social Hosting: Students over 21 provide alcohol to minors at parties/get togethers Social Access & Availability Prevalent Drinking Off Campus Residences Social Hosting: Students over 21 are not award of social hosting laws Perceived Risk Students perceive low risk of getting caught in own apartment Students perceive low risk of harm in own apartment Inconsistent Enforcement Negative Consequences Property management rules vary and do not get enforced Driving after Drinking Sexual Assaults Public Nuisance Property Damage Multiple jurisdictions and enforcement varies Social Norms Underage drinking is acceptable “College” norm to drink alcohol

21. Logic Model: USC Problem Priority : Prevalent Drinking at Football Game Days Root Causes Contributing Local Factors Problem Tailgate Environment, social hosting, alcohol available with little monitoring Social Access & Availability Prevalent, High Risk Drinking On football Game Days Excessive amounts of alcohol and time periods available for drinking Insufficient Laws and Policies Tailgating restrictions are inadequate Policies on tailgating vary due to mix of university/privately owned properties Inconsistent Enforcement Negative Consequences Large crowds vs. enforcement resources Underage Drinking Binge Drinking Driving after Drinking Black Outs Alcohol Poisoning Injuries Public Intoxication Property Damage Identification of illegal acts difficult to manage in large crowds Social Norms Alcohol central to tailgating culture Perception of “no rules, no consequences” on Game Days

22. USC Problem Priority : High Risk Drinking in Bars Root Causes Contributing Local Factors Problem Inconsistent Service Practices (ID checking, over-service) Access & Availability High Risk Drinking in Bars Extended Drink Hours Promotion Inconsistent enforcement of pricing, drink specials, and happy hour laws. Social Networking and Social Media Advertising Perceived safety: Safety risks are low with friends Negative Consequences Perceived getting caught: Easy to get in bars using fake ID and drink underage Binge Drinking Underage Drinking Black Outs Alcohol Poisoning Injuries Sexual Assaults Property Damage Arrests Social Norms Drinking to intoxication is acceptable Underage drinking is acceptable

23. Bar Sales and Services Subcommittee Long Term Goal By March 2016, reduce high risk drinking at bars by 10%. Objectives Reduce access and availability, address alcohol promotion through social media, increase perception of risk (harm and getting caught), and change social norms. Strategies/Activities Collect data on sales and service practices, promotion and advertising through social media, in order to develop and implement a responsibility toolkit and provide merchant education.

24. Off Campus Safety Committee Long Term Goal By March 2016, reduce high risk drinking at off campus properties by 10%. ObjectivesDecrease availability and accessibility, increase perception of harm and getting caught, consistently enforce policies and laws, improve referrals, and change social norms. Strategies/Activities Collect data from off property managers, courtesy officers, and students to develop an incentive package, offer education and programming, and improve collaboration with properties, law enforcement and university.

25. Other High Risk Substance Use Subcommittee Long Term Goal By March 2016, reduce high risk drinking at off campus properties by 10%. ObjectivesDecrease availability and accessibility, increase perception of harm and getting caught, consistently enforce policies and laws, improve referrals, and change social norms. Strategies/Activities Collect data from off property managers, courtesy officers, and students to develop an incentive package, offer education and programming, and improve collaboration with properties, law enforcement and university.

26. Community Organizing and Advocacy Subcommittee Long Term Goal By March 2016, reduce high risk drinking at off campus properties by 10%. ObjectivesDecrease availability and accessibility, increase perception of harm and getting caught, consistently enforce policies and laws, improve referrals, and change social norms. Strategies/Activities Collect data from off property managers, courtesy officers, and students to develop an incentive package, offer education and programming, and improve collaboration with properties, law enforcement and university.

27. Game Day Task Force Long Term GoalBy March 2016, reduce prevalence of drinking on game day by 5%. ObjectivesReduce social access and availability, consistently enforce policies and laws, and change social norms. StrategiesCollect data on tailgate policies and student drinking patterns on game day to inform and improve law enforcement and collaborations.

28. University of South Carolina Student Life Data, Assessment, and Evaluation Subcommittee Developed the Comprehensive Assessment Tool Kit, a data collection protocol packet for each subcommittee Purpose: to assist the subcommittees with conducting assessment specific to their priority area Assessment Tool Kit included: • brief overview of assessment process • data collection guidelines • variety of data collection methods to consider • general timeline template to assist with planning • tools and resources to use for data collection

29. Subcommittee Action Steps • Review Assessment Tool Kit • Select most appropriate data collection methods • Choose or develop data collection tools and protocols • Plan the data collection process and timeline • Implement the data collection process

30. Review Assessment Toolkit

31. Consider Collection Methodsand Timelines Subcommittees considered the following: • Key Informant Interviews • Focus Groups • Written Surveys • Community Forums • Environmental Scans • Social Media Analysis • Policy Review • Benchmarking

32. Select or Create Data Collection Tools and Protocols Data, Assessment and Evaluation Subcommittee developed the following tools: • Written surveys • Focus group questionnaires and protocols • Key informant interview questionnaires and protocols • Social Media Collection Tools • Environmental Scan Surveys

33. Example:Bar Sales and Services Subcommittee • Development of Tools/Instruments to assess Over-service & Effectiveness of Responsibility Toolkit: • Establishment Observation Tool, Volunteer Observer Protocol, and Establishment Observation Supervisor Checklist; • Utilization Assessment Tool; and • Patron Survey

34. Responsibility Toolkit

35. Toolkit Objectives: Patron • Raise awareness of negative consequences associated with drinking & driving • Increase perception of risk of getting arrested for DUI • Raise awareness of SC law prohibiting over-service to intoxicated patrons • Raise awareness of transportation optionsfor intoxicated patrons • Raise awareness of increased DUI enforcement

36. Toolkit Objectives: Employees & Management • Raise awareness of SC law prohibiting over-service to intoxicated patrons • Raise awareness that law enforcement is tracking place of last drink • Raise awareness of civil liability of establishment in event of tragedy • Raise awareness of increased DUI enforcement • Raise awareness of transportation options for intoxicated patrons • Increase self-efficacy to refuse service to intoxicated patron

37. Toolkit Objectives: Management (behavior) • Increase implementation of model policies governing alcohol sales • Increase establishment accountability to prevent driving after drinking • Increase partnerships between alcohol retail establishments & Task Force • Increase participation in responsible beverage service training (PREP)

39. Observational Scanning Findings • Patron/staff ratio • Drink specials/promotions • Age verification systems • Inconsistent systems to designate underage patrons vs. 21 and older • Staff drinking with patrons • Patrons requesting non-alcohol drinks ridiculed, hassled • Retailers see over-service as a personal responsibility issue, not a retailer issue

40. Conclusions from the Assessment Community Collaboration: • Challenging issue – multiple systems, sectors • Difficulty engaging alcohol retailers in substance use prevention efforts • “No one size fits all” – community demographics Responsibility Toolkit: • No good evidence of effects Observation Tool: • Provided documentation for both (a) pre intervention baseline and (b) post intervention • Manageable and easy to administer • Observer agreements (reliability) from 70-99%--EXCELLENT

41. Implications for Future Coalition Prevention Efforts • Continue assessment activities (bar sales and services observations, key informant interviews with bartenders and shuttle drivers; frequent, brief student surveys, etc.) • Continue collaboration efforts, particularly with retail establishments • Establish a student substance abuse cost analysis for key stakeholders • Utilize science concerning key variables and effective interventions • Establisha city-college data monitoring system (MIS)—reflect USC plus city. • Publish article “Observational Documentation in Evaluating Alcohol Over-service in Bars and Restaurants” for professional prevention journals to share findings with other college communities

42. Questions & Answers Rhonda DiNovo dinovo@mailbox.sc.edu Phone- (803)-777-3933