Driving survey administration training Betsy App  Lauran Arledge

1. Driving survey administration training Betsy App  Lauran Arledge Prevention Evaluation Partners  OMNI Institute

2. Introduction • Welcome Persistent Drunk Driving (PDD) Providers! • We have continuing and new providers with us today! • Objectives for Training: • Introduce the new PDD statewide outcome measurement tool – the “Driving Survey”. • Outline administration of the “Driving Survey”. • Review entry of “Driving Survey” data into ETO. • Learn about PEP data audit process and reports. • Optional supplemental Youth NOMs training offered the second part of the training.

3. Today’s Training • Please mute your phones to limit feedback and background noise • Press *6 to mute and unmute your phones • We have planned time for questions, but please feel free to unmute your phone to ask a question or let us know if you are experiencing any technical difficulties • All documents referenced in today’s training (including this presentation) are located on the DBH Portal in the PEP/ETO library • http://collaboration.omni.org/sites/dbh/Pages/default.aspx

4. Where Did the “Driving Survey” Come From? • Researchers at OMNI combined items from surveys used in academic research on the prevention of drinking and driving (Fromme, Katz, & Rivet, 1997; McCarthy et al., 2006). • The Driving Survey can be reviewed on the DBH Portal: http://collaboration.omni.org/sites/dbh/Pages/default.aspx

5. THE SURVEY INCLUDES • Demographic items (age, sex, race, ethnicity) • Items that measure drinking and driving behaviors(items 7-14) • Items that measure attitudes(items 15-27). The items on the Driving Survey have been validated Researchers have scientifically examined the items and determined that they measure what they're supposed to measure: attitudes and behaviors around drinking and driving.

6. Why Are We Using the Driving Survey? • Statewide Outcome Measure • Allows DBH to measure outcomes across multiple programs in the state that are similar. This effort allows us to see the average impact of PDD-funded programs on participant outcomes across Colorado. • Program Impact • Allows you to see program outcomes; if your program targets the things measured by the survey (perceptions, attitudes and behaviors around drinking and driving), you can expect to see positive change from pre-test to post-test. • Grant Requirement • DBH is required to report to the state outcomes related to PDD funding • required statewide pre/post measure for the Division of Behavioral Health’s (DBH) who serve participants ages 15-34.

7. Consent • Active Consent • Parents/guardians must sign form to allow their child/children to participate. • Active Consent Template can be found on the DBH Portal: http://collaboration.omni.org/sites/dbh/Pages/default.aspx titled “Active Consent If Entering ETO Assessments” • Passive Consent • Parents/guardians sign form only if they object to their child’s/children’s participation. • Passive Consent Template can be found on the DBH Portal: http://collaboration.omni.org/sites/dbh/Pages/default.aspx titled “Passive Consent If Entering ETO Assessments”

8. Survey Administration The demographic information collected here is used for reporting purposes.

9. Step One: • Assign a unique local ID to each participant • Keep a log of your local ID assignments • See DBH Portal for “Instructions for Assigning ID Numbers” document • Make sure to keep track if it a pre-test or post-test (front page) • Contact your PEP TA provider for the Spanish-language version of the survey Tip for success: Keep a notebook or excel file with Local ID assignments • Step Two: • Administer Driving Survey to participants • See “Survey Administration Tips” document on the DBH Portal http://collaboration.omni.org/sites/dbh/Pages/default.aspx

10. Step Three: • Enter Driving Survey data into ETO • Enter one assessment into ETO for each participant • Make sure to enter the six-digit local ID into ETO correctly • Make sure the data entered into ETO matches exactly the participants’ responses on the paper versions of the survey • Tip for success: Create a system for noting which surveys have been entered into ETO, such as the staff member who enters the data into ETO writes his/her initials in the upper left hand corner of the paper version of the Driving Survey • Step Four: • Double check all data entered into ETO using the “Driving Survey Data Submission Checklist” document on the DBH Portal http://collaboration.omni.org/sites/dbh/Pages/default.aspx

11. Step Five: • Submit Driving Survey via mail • Make photocopies of each completed Driving Survey and keep photocopies securely filed on-site • Fill out “DBH Survey Mailing Summary Form” document on the DBH Portal http://collaboration.omni.org/sites/dbh/Pages/default.aspx Tip for success: Make sure each batch mailed to OMNI is accompanied by a DBH Survey Mailing Summary Form • Mail surveys to OMNI prior to data submission deadline • Deadlines are communicated through DBH Portal and PEP email account (DBHstatewideeval@omni.org)

12. Entering Survey Results into ETO • DBH Portal: http://collaboration.omni.org/sites/dbh/Pages/default.aspx

13. Data Audits and Reports • Data Audits: • Goal: High quality data for state-required reporting on prevention work in Colorado. • The PEP Analysis team systematically reviews data entered into ETO to surface any data entry errors that maybe present • Quarterly data audit process: • Your PEP TA provider will send you an excel file with any data entry errors that require correction • Driving Survey Report will be available soon so you can check your own work easily

14. Conclusion • Thank you for your time! • Please visit the DBH Portal PEP/ETO Library. • Your PEP TA provider can help with any questions related to your evaluation process! • Contact Information: • Betsy App: bapp@omni.org • Analysis Lead for OMNI Prevention Evaluations Partners • Lauran Arledge: larledge@omni.org • TA Lead for OMNI Prevention Evaluations Partners

15. Youth Noms Administration supplemental Training Prevention Evaluation Partners  OMNI Institute

16. Introduction • The Youth NOMs survey is: • a 26-question survey • required statewide pre/post measure for the Division of Behavioral Health’s (DBH) SAPT block grant, PDD, and LEAF grantees who serve youth ages 12-18.

17. Why Are We Using Youth NOMs? • Youth NOMs survey is designed to measure several different concepts that the Center for Substance Abuse Prevention (CSAP) has designated as a national outcome measures including: • lifetime usage of alcohol, tobacco, and other drugs (ATOD) • past 30-day use of ATOD • attitudes toward ATOD use • perceived risk of harm from ATOD use • drunk driving behavior • communication with parents • receptivity to workplace drug testing • exposure to prevention-messaging advertising

18. Consent • Active Consent • Parents/guardians must sign form to allow their child/children to participate. • Active Consent Template can be found on the DBH Portal: http://collaboration.omni.org/sites/dbh/Pages/default.aspx titled “Active Consent If NOT Entering ETO Assessments” • Passive Consent • Parents/guardians sign form only if they object to their child’s/children’s participation. • Passive Consent Template can be found on the DBH Portal: http://collaboration.omni.org/sites/dbh/Pages/default.aspx titled “Passive Consent If NOT Entering ETO Assessments”

19. Step One: • Order Scantron forms from OMNI • Providers are responsible for ordering Scantron forms on a yearly and as- needed basis. • Placing an order is done through DBHStatewideeval@omni.org email using the “Youth NOMs Scantron Order Form” found on the DBH Portal http://collaboration.omni.org/sites/dbh/Pages/default.aspx • Contact your PEP TA provider for the Spanish-language version of the survey • Step Two: • Assign a unique local ID to each participant • Keep a log of your local ID assignments • See DBH Portal for “Instructions for Assigning ID Numbers” document • Make sure to track if the survey is a pre-test or post-test. Tip for success: Keep a notebook or excel file with Local ID assignments

20. Step Three: • Administer Youth NOMs surveys to participants • See “Survey Administration Tips” document on the DBH Portal http://collaboration.omni.org/sites/dbh/Pages/default.aspx • Instruct participants to use a #2 pencil and fill in the bubbles on the Scantron completely • Step Four: • Prepare Scantrons for mailing • Please do not staple Scantrons together (use paperclips) • Review each completed Scantron and check each one to make sure: • Participant’s birth date is filled out properly (e.g., birth date does not indicate the current year) • Administration date (labeled “Today’s Date” on the Scantron) is filled out properly • Stray marks are erased completely

21. Step Five: • Submit Youth NOMs Scantrons via mail • Use “Youth NOMs Submission Checklist”(DBH Portal) to ensure all steps are completed. • Make photocopies of each completed Scantron and file photocopies securely on-site. • Please do not staple Scantrons together • Fill out the “DBH Survey Mailing Summary Form” completely and accurately. • Include this form with each batch of completed Scantrons (originals) to be mailed flat (no folds/creases) to OMNI • Make sure that each batch mailed to OMNI is accompanied by a Mailing Summary Form. This ensures that your data can be included in your outcome report and the overall state outcome report! • Mail Scantrons to OMNI prior to data submission deadlines • Deadlines are communicated through DBH Portal and DBHstatewideeval@omni.org

22. A Note on Youth NOMs and ETO • You are not required to enter Youth NOMs into ETO but you may do so if you wish. • Youth NOMs is simply entered into ETO as another assessment and can be selected by selecting if from the drop down menu. • The benefit of entering Youth NOMs into ETO is that you can pull reports to review your data.

23. Conclusion • Thank you for your time! • Please visit the DBH Portal PEP/ETO Library. • Your PEP TA provider can help with any questions related to your evaluation process! • Contact Information: • Betsy App: bapp@omni.org • Analysis Lead for OMNI Prevention Evaluations Partners • Lauran Arledge: larledge@omni.org • TA Lead for OMNI Prevention Evaluations Partners

24. DBH Portal Documents The following is a list of document titles that can be hound on the DBH Portal, PEP/ETO Library.

26. ETO Data Entry Manual for Driving Survey Data Entry

27. Click here to begin entering the Driving Survey

28. You can search by last name or leave blank to bring up all active participants.

29. Select the participant by clicking on their name.

30. Click here.

31. Click here to select Driving Survey.

32. Enter survey administration date here (not the day that it is entered into ETO. Enter the Unique 6-digit local ID here. Enter data into ETO exactly as it appears on the paper survey.

33. Click here to save assessment.

34. Click here to continue.

35. Click either “Edit” to make changes to your entry or “Continue” to save your entry.