Survey Methodology Survey Instruments (1)

1. Survey MethodologySurvey Instruments (1) EPID 626 Lecture 7

2. What is a survey instrument? • For interviews: A script for interviewers, including introductions, instructions, and questions • For self-administered surveys: A questionnaire

3. Essential components for developing a survey instrument • Statement of purposes • What do you want to accomplish with the survey? • List of the variables to be measured • Group into logical categories • Draft analysis plan • What are your dependent, independent, check variables?

4. Preliminary question design steps • Focus groups • Draft questions • Cognitive laboratory interviews

5. Focus groups • Objective: To compare the reality about which respondents will be answering questions with the abstract concepts embedded in the study objectives (Fowler, 1995) • Usually 6-8 people per group; have a couple of groups

6. Focus groups(2) • Try to get participants who are similar to your survey population • Talk about perceptions, experiences, and feelings related to survey subject

7. Drafting questions • Look for standard questions from other surveys • Benefit from their experience • Enhance comparability of your survey with previous surveys • Write good questions (see next lecture)

8. Cognitive laboratory interviews • Respondents are brought into a laboratory setting • May be videotaped • Interviews conducted by cognitive psychologist or experienced investigator • Goal: To get information about how the respondent understood the questions and about the way they answered them

9. Layout of survey instrument • Make it clear and simple • Usually begin with straightforward questions, put complicated or sensitive questions in the middle or at the end

10. Interviews • Should have everything scripted, including introductions, instructions, transitions, definitions, and explanations • Be very explicit about skip instructions • Differentiate instructions and optional wording from questions • Usually done with all caps and parentheses, respectively

11. Self-administered • Questionnaire should be self-explanatory (minimal instructions needed) • Limit to closed questions • Use short questions with consistent formats • Minimize skips; make them very clear

12. Field pretesting • Goal: Find out how well the data collection protocols and survey instruments work under realistic conditions

13. Interviews • Usually interview 20-50 respondents who are similar to your survey population • Interviewers note ways to improve survey instruments

14. Quantitative methods for pretesting • Ask interviewers to fill out a rating for each question • Is it easy to read as worded? • Do respondents understand the question in a consistent way? • Can respondents answer the question accurately?

15. Quantitative methods for pretesting(2) • Taping and behavior coding • Trained coders review the recordings and evaluate the question and answer process in a consistent way • Do the interviewers read the question as worded? • Do the respondents ask for clarification? • Do the respondents initially give an inadequate answer that requires interviewer probing?

16. Self-administered • Have respondents (who are similar to your survey population) fill out the survey and then discuss • Were the instructions clear? • Were the questions clear? • Were there any problems understanding what kind of answers were expected?