Research Support Services Online Survey Development: Tips and Pitfalls

1. Research Support Services Online Survey Development: Tips and Pitfalls Jennifer McGugan Research Support Specialist mcgugj@mcmaster.ca October 24, 2012

2. Surveys Many study designs use surveys to collect information Online or web-based surveys are an increasingly more popular way to collect survey information

3. “Web surveys make feasible the delivery of multimedia survey content to respondents in a standardized way using self-administered methods. This opens up a whole new realm of survey possibilities that were heretofore impossible or extremely difficult to implement using more traditional methods.” (Couper, 2000)

4. Purpose Discuss tips and pitfalls of survey development with an online focus Not instruction on technical aspects of online survey software

5. Before You Begin Clearly identify your research question Is a questionnaire or survey the best way to answer your question? Are there any existing tools or questionnaires that you can use to answer your research question? Valid Reliable Justify why a new questionnaire needs to be developed

6. Survey Development “The design of a questionnaire must consider how to motivate someone who receives it to become a respondent and complete the survey.” (Dillman, 2009)

7. Design Create the most appealing appearance for respondents to complete your online questionnaire Carefully consider the questionnaire design

8. Tips and Pitfalls Create informative welcome and closing screens written in a friendly, professional manner Welcome Screen: - title, purpose, instructions, participant rights , study contact information and consent Closing Screen: - message to the respondent regarding completion of the survey and acknowledgement of respondent’s contributions

9. EXAMPLE: MCMASTER FITNESS CENTRE EVALUATION We are inviting you to participate in this survey, led by Jane Smith, PhD, at McMaster University, as well as a follow-up survey six months from now. All adult members of the McMaster Fitness Centre have been invited to participate. Individuals who complete both the evaluation survey and the follow-up survey will be entered into a draw to win an ipad as a token of our appreciation. The survey questions will take less than 10 minutes to complete. The intent of this evaluation is to determine if, and the extent to which you use the exercise facility and how that contributes to your health. All responses will be kept confidential and anonymous. Anonymity of data will be maintained so that any reports produced from the study will not provide identifying information. This information will be kept in the project coordinator's office with electronic files on a computer that is password protected, so that only research staff will have access to this data. There are no foreseen risks in participating in this study. If you choose not to participate in this study there will not be any negative consequences. You may also choose to withdraw from the survey at any time, or choose not to answer specific questions.   The results of the study can be made available to you, or you may request notification of any publications produced as a result of this study by contacting the research coordinator. If you have any questions or concerns please contact Jennifer McGugan at mcgugj@mcmaster.ca or 905-525-9140 ext 23220.    This study has been reviewed by the Hamilton Health Sciences/McMaster Faculty of Health Sciences Research Ethics Board (HHS/FHS REB). The REB is responsible for ensuring that participants are informed of the risks associated with the research, and that participants are free to decide if participation is right for them. If you have any questions about your rights as a research participant, please call The Office of the Chair, HHS/FHS REB at 905.525.2100 x. 42013.   I have read the preceding information thoroughly. By clicking the submit button, I agree to participate in this study.

10. EXAMPLE: THANK YOU! Your answers have been received. Thank you for taking the time to complete this survey. We value your contribution. Sincerely, Jane Smith

11. Introduce your question groups with a description to inform participants about the next steps. EXAMPLE: MCMASTER FITNESS CENTRE EVALUTION The next group of questions ask about your exercise habits. Please answer the questions while keeping in mind your regular weekly exercise habits.

12. Order questions logically and group by content EXAMPLE : 1. What exercise equipment do you use? Select all that apply. Treadmill Exercise Bike Elliptical Machine Rowing Machine Free Weights Weight Machines Other Please Specify: 2. How many times per week do you use the McMaster Fitness Centre? # of times 3. What is your occupation? Yes No 4. Have you ever had surgery? Yes No

13. Questions should apply to your respondents EXAMPLE: 5. If you regularly use the treadmill, how long do you spend on it? minutes Specify the concept EXAMPLE: 6. How many times did you exercise? Number

14. Ask one question at a time EXAMPLE: 7. Do you have and use a step tracker? Yes No Use complete sentences with as few words as possible and keep them simple and familiar EXAMPLE: 8. Do you feel like you have more doggedness after using the MUFC more than 3 times per week? Yes No

15. Create mutually exclusive response categories EXAMPLE: 9. Please select the age category applies to you: 18-30 30-45 45-60 60 and above

16. Have a purpose for open-ended questions EXAMPLE: 10. How do you feel about exercise?

17. Use mandatory questions only if necessary Put more sensitive questions toward the end of the survey Include instructions to ensure respondents know how to answer your questions Number questions consecutively See Dillman (and others) for many other things that should and should not be done when writing questions and designing a questionnaire .

18. Online Survey Software Survey Monkey Fluid Surveys Checkbox Survey Crafter LIME SURVEY – Used by McMaster University

19. Question Formats

20. Single choice questions – List (radio)

21. Arrays – 5 point choice

22. Mask questions - Date

23. Multiple numerical input

24. Text questions

25. Multiple choice

26. Question editing

27. Survey procedures Email invite to participate in survey tokens vs. self-identified ids date stamps IP address Reminder email(s) depending on study protocol Frequency Survey open and close dates Duration

28. Research Support Services School of Nursing Research Support Services can assist you with your online survey on a fee-for-service basis Getting Started Complete a request form on our website Set up a contract with Susanne King kings@mcmaster.ca

29. Research Support Services QUESTIONNAIRE Your questionnaire in a complete form with all descriptions and introductions Welcome, closing and email messages, if applicable Are you going to be pilot testing your questionnaire? Content validity Readability If you are using an existing questionnaire, have you obtained the necessary permissions?

30. Research Support Services PARTICIPANTS Complete contact list of participants or list serv email addresses If you have a participant gift or prize how are you distributing it? What is your timeline? Reminders, close date, etc. Are there follow up surveys that need to be matched Tokens vs. Self-identified IDs

31. Research Support Services DATA Do the data need to be anonymized? Will you need to do any recoding? In which format would you like to receive your data? Excel SPSS

32. Couper, M. (2000) WEB SURVEYS A REVIEW OF ISSUES AND APPROACHES Public Opinion Quarterly Volume 64:464–494. Dillman, D. (2009) Internet, Mail and Mix-mode Surveys: The Tailored Design Method

33. Thank youQuestions?