Implementing the ACHA-NCHA: What users need to know to get started

1. Implementing the ACHA-NCHA: What users need to know to get started Mary Hoban, Ph.D., E. Victor Leino, Ph.D., Theresa Jackson Hughes, MPH American College Health Association

2. What will we cover today? • Overview of the ACHA-NCHA surveying process • Steps in the surveying process • Paper vs. Web versions • Incentives • Sampling • Scheduling • Reporting • Much more! • Additional Resources • Question and Answer Session

3. Summary of steps involved in the surveying process • Determine the purpose of surveying and target population • Identify feasibility and sources of support/campus resources • Determine cost • Specify the method for conducting the survey • Determine the incentives to be used, if any • Determine sampling procedures and level of over-sampling needed

4. Summary of steps involved in the surveying process • Specify the schedule for conducting the survey and the amount and schedule of follow-up contacts • Determine if you want to add extra questions or automatically downloaded variables • Obtain Institutional Review Board and/or administrative approval • Send ACHA all required information • Determine how you will report results to other campus officials

5. Purpose of Surveying Pre-test and/or Post-test General Health Assessment Needs Assessment Comparison to National Sample Program Planning In what population are you interested? Freshmen Athletes Graduate Students Commuters All Students Other? Purpose of Surveying and Target Population

6. Feasibility, sources of support, and campus resources • Is it feasible to conduct the ACHA-NCHA on your campus? • Depends on sources of support and campus resources available • Sources of support • Administrators • Health Center staff • Campus Recreation staff • Other Student Affairs staff • Information Technology staff • Campus resources • Graduate students • Academic departments

7. Cost • Cost • ACHA costs available at: www.acha.org/projects_programs/NCHA_Pricing.pdf. • Other costs may include personnel costs for your institution, postage, and incentives

8. Cost

9. Estimate ACHA Institutional Member Price Based on 3,000 Contacts

10. Methods of surveying • Paper Survey Options • All students • Randomized mailed survey • Randomized classroom survey • Campus assessment day • New student orientation • Web-based Survey Options • All students • Randomized NCHA-Web survey

11. Paper Survey Options

12. Randomized Classroom Survey • Cooperation from instructors is critical • Selection of classes • Random • Should be representative of the universe from which you are drawing your sample • Seek assistance of registrar’s office • Approximately 35 minutes to complete • Complete steps available in the User’s Manual

13. Advantages Convenient Cost-effective High response rate Possible to collect information from a wide coverage of majors if classes selected randomly Disadvantages Instructors may be reluctant to give up class time for survey purposes Classes may not be as representative of various majors and campus demographics as other methods Randomized Classroom Survey

14. Randomized Mailed Survey • Randomized selection of students (or all students) • Send survey with an addressed stamped envelope • May send thank you letters and re-contact letters

15. Advantages Ease of randomization Greater generalizability of results Follow-up notices possible, which may increase response rate Well documented procedures and policies Disadvantages Expense (costs of mailing, tracking, and re-contacting) Require more administrative work Generally has a response rate below 50% Randomized Mailed Survey

16. How to avoid pitfalls of paper-based surveying • Get organized early! • Classroom based survey • Secure approval of instructors as soon as possible and remind them of when you will be coming • Bring plenty of pencils or pens • Mailed surveys • Mail to campus address, not permanent address • Send follow-ups to increase response rates • Pay for all return postage!

17. Web-based Survey Options

18. Randomized NCHA-Web Survey • Hosted through a secure server • Students are generally invited to participate via e-mail • IHE sends a list of e-mail addresses ACHA • Can be randomized by registrar, randomized by ACHA, or can invite entire population of students • Less work on the surveyor

19. Advantages Cost effective Non-responders can be contacted easily Respondent may scroll through survey Possible for the respondent to leave and come back to the survey Easy to select those who will receive incentives Ease of administration Easy to add extra questions Disadvantages Students may not trust web security and may choose not to participate Low response rates Typically between 10-50% Average: 25% Issues with spam blockers, faulty e-mail addresses Randomized NCHA-Web Survey

20. How to avoid pitfalls of web-based surveying • Obtaining student e-mail addresses • Spam filters and rate control systems

21. Incentives • Used to increase response rates and thank students for participating • Methods of giving incentives • Randomly select one participant to receive a large incentive • Randomly select multiple participants to receive several smaller incentives • Give incentives to all participants

22. Incentives • Example incentives • iPod • Gift certificates • Bookstore • Local restaurants or stores • Cash • University parking permit • Airline tickets • DVD player • Tuition • $2.50 Dining Services Coupon • Extra Credit

23. Sampling procedures • Goal is to select a representative sample of your institution • Usually not necessary to sample entire institution • Important to use random sampling • Randomized classrooms • Want to sample variety of academic subjects and levels

24. Sample Size

25. Sample Size and Response Rates • Mailed and web surveys have lower return rates (approximately 25%) • This affects the number of students you should invite to participate • For example, if the desired sample is 600 for a mailed survey, 2400 names should be randomly drawn to receive the survey • Better to oversample than undersample!

26. Scheduling • Pre-survey publicity • Spring version • Recommended to survey between the last week of March and the first week in May • More than 30 days after spring break and winter break • Fall version • Possible to survey at almost anytime • More than 30 days after fall break • Recontacts • Survey close

27. Extra Questions • Web version • Possible to add up to five additional questions for a surcharge of $700 • Added to end of the online survey • Reposes automatically downloaded with other ACHA-NCHA data • Must be single questions • Each item on a “Select all that apply” question is considered as a separate question • May not have more than 10 response categories • Paper version • Tracked by lithocodes on paper ACHA-NCHA • No cost nor limits • Campus does all the work

28. IRB/Administrative Approval and Informed Consent • The ACHA-NCHA is research, and as such, all participants must be protected • Institutional Review Board • Initiated to protect human subjects from potential harm due to the nature of the research • If your school does not have an IRB, you must secure human subject protection and approval to administer the survey from an administrator who is familiar with these procedures • Informed consent • Each IHE should conform to the institutional requirements for informed consent and human subject’s protection • Sample consent forms are provided in the User’s Manual • Each IHE must provide IRB/administrative approval to NCHA prior to administration of the survey

29. What do I need to send to ACHA to implement the NCHA paper version? • Before Implementation • The ACHA-NCHA Order form • Initial payments for both the total number of student contacts and report package • IRB/Administrative approval letter • After Implementation • Completed surveys (do not bend) • Demographic information sheet • Final payments

30. What do I need to send to ACHA to implement the ACHA-NCHA-Web? • Before Implementation • The ACHA-NCHA Order form • Initial payments for both the total number of student contacts and report package • Spreadsheet file of student e-mail addresses in the first column of the file and any personalization/download variables in columns thereafter • The letter of invitation/consent and subject line • The reminder letter of invitation/consent and subject line to be sent to non-responders • Dates for initial contacts, recontacts, and survey close • IRB/Administrative approval letter • After Implementation • Demographic information sheet • Final payments

31. Reporting Results • ACHA provides the following products: • Institutional data file on CD • Institutional data report • Institutional executive summary • Reference group data report • Reference group executive summary

32. Data Report Example

33. Suggested Resources • Neuman, W. Lawrence (2000). Social Research Methods: Qualitative and Quantitative Approaches, 4th ed. Boston: Allyn and Bacon. Chapter 10. • Babbie, Earl (1998). The Practice of Social Research. Belmont: Wadsworth. Chapter 10. • ACHA-NCHA User’s Manual • Available at www.acha.org/projects_programs/ACHA-NCHA_USERS_ MANUAL_ver2-004.pdf • ACHA-NCHA Frequently Asked Questions • Available at www.acha.org/projects_programs/ncha_webfaqs.cfm

34. Questions about the ACHA-NCHA? • Contact: Mary T. Hoban, Ph.D., CHES American College Health Association 410.859.1500 mhoban@acha.org