Surveys

1. Surveys The most widely used data-gathering device in sociology

2. Surveys • Survey research rests on sampling. • A sample is simply the collection of people from whom you are gathering data. • There are tons of ways to sample, but we want to be sure that we sample with generalizabilty in mind.

3. Surveys • What type of sampling does the General Social Survey use??? • In 2004, it was a multistage cluster sample. It is a full probability sample of US households, meaning that each house had an equal chance of being selected. • Persons in households were selected systematically to make demographic characteristics of the sample equivalent to the characteristics of the U.S. population.

4. Surveys • What are the implications of the General Social Survey’s sampling??? • The GSS is an adults-only survey of persons in households. Therefore, it underrepresents: • 18 – 24 year-olds (many not living in households—military, college, roaming) • 65 and over (many not living in households—vacations, RVs, assisted living) • Persons who live in large households (only one person per household is interviewed) • Homeless and some poor (not in official households, in shelters, on streets)

5. Surveys • Sampling is a technique for selecting subjects. So what is a survey? • A process of presenting a standard series of questions to a sample of persons. • The purpose is to accurately reflect the beliefs, attitudes, and behaviors of the sample, so that one can generalize accurate information to some larger population. • The survey is the most widely used technique in sociology because it is best suited for looking at the complex social world. To capture that world accurately, we have to measure it “in situ.” That means we must sample from it, and measure many different phenomena, where they are found naturally.

6. Surveys • Types of Surveys: • The survey can be administered as a(n): • Written document filled out by the sample’s respondents • alone • in groups • or individually in group settings (group administered) • Face-to-face interview, with questions read by an interviewer • Telephone interview • Electronic Questionnaire

7. Surveys • Survey Delivery: • Written Questionnaires: • Mail • Group Administered Questionnaires • Drop-off Surveys • Electronic Questionnaires: • Distributed as E-mail • Web pages—sampling can be done via normal methods, but often aren’t • Public computers in high-traffic areas • In the future: • Files sent via mail, replacing written surveys • Mixed formats offered to subjects (choose: pen-and-paper, web page, file on disk)

8. Surveys • How is the General Social Survey Delivered and Administered??? • The GSS is a one-on-one interview. An interviewer asks subjects questions. • In 2000, the interviewer used paper and pencil. • In 2002 they switched to computer assisted personal interviewing (CAPI). A computer program provided questions and interviewers punched in answers, much like in telephone polling with CATI.

9. Surveys • Before Choosing a type of survey, you should consider four (4) key issues: • Respondent Attitude: Think like a subject. How will they react to the way they are approached? • Nature of Questions: Open-ended and scale questions would be difficult over the phone. You have to consider whether respondents can adequately answer your questions via your medium.

10. Surveys • Before Choosing a type of survey, you should consider four (4) key issues: • Cost: What can you afford? Target population, geographic distribution, and type of questionnaire all affect costs. • Generalization needs: If you need a broad sample, you can’t do drop-off surveys. If you want to sample from the elderly, you should avoid electronic surveys as currently conceived.

11. Surveys • So you have to think about the strengths and weaknesses of your survey delivery method. • In a minute you’ll break into five groups to discuss strengths and weaknesses of: • Mailed • Group Administered • Face-to-Face Interviews • Telephone • Electronic • I’ll give you an example using the dropoff:

12. Surveys • Drop Off • Strengths versus Weaknesses

13. Strengths Convenient for respondents, like mail survey. Potentially higher response rates than mailed survey because the researcher has a presence (if dropped off at residence) or organizational endorsement (if dropped off at an organization). Weaknesses Time. Because of the personal contact, this uses more researcher time. Challenges to generalizability— Hard to get a broad sample. Target population must be pretty specific because of the limited ability to make contact with different people. Lack of control over distribution. Lower response rates than face-to-face and telephone (possibly) Inaccuracy—Behavioral problems in groups, “playful” responses. Surveys—Drop Off

14. Surveys • So you have to think about the strengths and weaknesses of your survey delivery method. • Break into Five (5) groups. • Mailed • Group Administered • Face-to-Face Interviews • Telephone • Electronic • Spend twenty (20) minutes generating strengths and weaknesses of your group’s survey delivery type.

15. Surveys • You must weigh the strengths and weaknesses of the survey delivery methods to choose the one that is appropriate and feasible. • The following slides consider the strengths and weaknesses of five delivery methods (the lists are not exhaustive).

16. Surveys • Mailed • Strengths versus Weaknesses

17. Strengths Cost: The greatest cost in most businesses is wages. Mailed surveys reduce labor compared with face-to-face or phone. However, human data entry is expensive—use scan forms. Low cost for respondent when postage paid envelope is provided Convenience for participants Reduced Bias—no interviewer Facilitates generalizability—Easy to reach more people over large area Better Geographic (and demographic) targeting Weaknesses Motivation, People may not be motivated to respond or respond thoroughly when researcher is not present Lower response rates, compared with face-to-face or group administered Challenges to accurate response: Assumes physical skills Literacy language abilities This likely biases results. Interpretations of Questions can be inaccurate—bias from no interviewer to assist. Who fills it out? Surveys—Mailed

18. Surveys • Group Administered • Strengths versus Weaknesses

19. Strengths Higher response rate, especially if a captive audience such as school. Permits complex questions—Greater specificity and variation in question types because there’s someone to answer questions and explain. Good for educationally challenged and children: Someone is there to help. Fast way to get sample. Inexpensive, especially if you don’t hire many people. You get many cases at once. Data entry is still an issue. Weaknesses Challenges to generalizability—Hard to get a broad sample. Target population must be pretty specific. Scheduling. Requires a block of time convenient for all. Inaccuracy—behavioral problems in groups may distract respondents or even lead to “playful” responses Privacy—setting may allow others to view subjects’ responses Surveys—Group Administered

20. Surveys • Face-to-Face Interviews • Strengths versus Weaknesses

21. Strengths Permits complex questions—Greater specificity and variation in question types because there’s someone to answer questions, explain, and probe. Highest response rate. Personal appeal increases likelihood that respondents will participate. Convenience—Allows rescheduling of visits (helps with response rate too). Reduces disability bias—interviewer may assist respondents CAPI: If using computer entry of responses, interviewing doubles as data entry. Weaknesses Cost. Payroll, driving, and so forth are expensive compared with other forms. Increased Bias. The appearance and style of the interviewer can bias participation or answers. Imagine questions about drug use or sexuality. Interaction between interviewer and interviewee can lower reliability Oral format limits types of questions that you can ask, although visuals may be used. Interviewer safety is of concern and fear can lead to falsified questionnaires. Surveys—Face-to-Face

22. Surveys • Telephone Interviews • Strengths versus Weaknesses

23. Strengths Permits complex questions—Greater specificity and variation in question types because there’s someone to answer questions, explain, and probe. Not as good as face-to-face. Higher response rate (maybe) Personal appeal increases likelihood that respondents will participate. Not as high as face-to-face. Convenience—Allows rescheduling of contact (helps with response rate too). Fast—surveys can be quickly completed Facilitates generalizability—Easy to reach more people over large area, cheaper than face-to-face but more expensive than mailed survey Environmentally friendly compared with other methods except electronic Weaknesses Cost. Payroll is still an issue, but is cheaper than face-to-face. Number of questions is limited by low tolerance for being on the phone for long. Biases: The linguistic style of the interviewer can bias participation or answers. Imagine questions about drug use or sexuality. If using area codes for sampling frame in an era of cell phones. Type of household that answers the phone. Who has a phone? Limits on the types of questions that you can ask: relies on short-term memory and quick recall visuals are not possible Falling response rates because of American attitude toward phone. Americans hang up on strangers or screen calls Phone interruption is an annoyance Surveys—Telephone

24. Surveys • Electronic Surveys • Strengths versus Weaknesses

25. Strengths Cheap. Depending on type, data are automatically entered by respondents Ease of editing questionnaire and analyzing data Fast—Distribution and response time is reduced. Higher response rates (maybe) than other methods, sometimes. Reduced bias—You get more candid responses. One researcher found: Number of persons having sex with same-sex individuals doubles compared with mail and face-to-face surveys. Some complexity permitted. Interaction can be available, either through faqs, e-mail, or call to researchers. Environmentally friendly compared with other methods except telephone Respondents can view “tally” of responses Can reach samples on a global scale Convenient for respondents Weaknesses Selection Bias & Generalizability: Must have computer access, but can be used to generalize to computer-access population. Lacks complete sampling frame Email contact may be treated as SPAM Construction of questionnaire can be challenging—often need web or computer expertise Challenges to accurate response: Assumes physical skills Literacy and language abilities Instructions may get tedious and confuse respondents Require electronic literacy This likely biases results. Glitches can kill participation for some Long-term response rates may be no better than other survey forms. Who fills it out? The Web Culture encourages “hacking” and falsification of information Surveys—Electronic

26. Surveys—Electronic • Of the types of electronic survey, those that carry us toward the population coverage of mail surveys and the response rates of the face-to-face survey will be judged the best. • Current forms ranked from worst to best: • Public computers in high-traffic areas—as bad as man on the street convenience samples. • Web pages—sampling can be done via normal methods, but often aren’t • Distributed as E-mail • Hybrids will likely rule the electronic survey in the future.