Survey Delivery

1. Survey Delivery The survey is the most widely used data-gathering device in criminology

2. Survey Delivery • The survey questionnaire can be: • Written—paper and pencil--document filled out by the sample’s respondents • alone • in groups • or individually in group settings (group administered) • Face-to-face interview, with questions read by an interviewer • Telephone interview • Electronic format

3. Survey Delivery • Written—paper and pencil--questionnaires can be delivered by or as: • Mail • Group Administered Questionnaires • Drop-off Surveys • Electronic Questionnaires can be delivered as: • E-mail solicitation • Web pages—sampling can be done via normal methods, but often aren’t • Mixed formats offered to subjects (choose: pencil and paper, web page, file on disk)

4. Surveys • How is the General Social Survey Delivered and Administered??? • The GSS is a one-on-one interview. An interviewer asks subjects questions. • In 2000, the interviewer used paper and pencil. • In 2002 they switched to computer assisted personal interviewing (CAPI). A computer program provided questions and interviewers punched in answers, much like in telephone polling with CATI.

5. Surveys • Before Choosing a type of survey, you should consider four (4) key issues: • Respondent Attitude: Think like a subject. How will they react to the way they are approached? • Nature of Questions: Open-ended and scale questions would be difficult over the phone. You have to consider whether respondents can adequately answer your questions via your medium.

6. Surveys • Before Choosing a type of survey, you should consider four (4) key issues: • Cost: What can you afford? Target population, geographic distribution, and type of questionnaire all affect costs. • Generalization needs: If you need a broad sample, you can’t do drop-off surveys. If you want to sample from the elderly, you should avoid electronic surveys as currently conceived.

7. Surveys • So you have to think about the strengths and weaknesses of your survey delivery method. • In a minute you’ll break into five groups to discuss strengths and weaknesses of: • Mailed • Group Administered • Face-to-Face Interviews • Telephone • Electronic • I’ll give you an example using the dropoff:

8. Surveys • Drop Off • Strengths versus Weaknesses

9. Strengths Convenient for respondents, like mail survey. Potentially higher response rates than mailed survey when leadership is supportive. Otherwise, it would be lower. Weaknesses Time. Because of the personal contact, this uses more researcher time. Challenges to generalizability— Hard to get a broad sample. Target population must be pretty specific because of the limited ability to make contact with different people. Lack of control over distribution. Lower response rates than face-to-face and telephone (possibly) Inaccuracy—Behavioral problems when subjects collude and give “playful” responses. Surveys—Drop Off

10. Surveys • You must weigh the strengths and weaknesses of the survey delivery methods to choose the one that is appropriate and feasible. • The following slides consider the strengths and weaknesses of five delivery methods (the lists are not exhaustive).

11. Surveys • Mailed • Strengths versus Weaknesses

12. Strengths Cost: The greatest cost in most businesses is wages. Mailed surveys reduce labor compared with face-to-face or phone. However, human data entry is expensive—use scan forms. Low cost for respondent when postage paid envelope is provided Convenience for participants No interviewer bias Facilitates generalizability—Easy to reach more people over large areas Better Geographic (and demographic) targeting Weaknesses Motivation, Unmotivated to do work for unknown researcher Lower response rates, compared with face-to-face or group administered Challenges to accurate response: Assumes physical skills Literacy language abilities This likely biases results. Interpretations of Questions can be inaccurate—bias from no interviewer to assist. Who fills it out? Surveys—Mailed

13. Surveys • Group Administered • Strengths versus Weaknesses

14. Strengths Higher response rate, especially if a captive audience such as school. Permits complex questions—Someone can answer questions. Good for educationally challenged and children: Someone is there to help. Fast way to get sample. Inexpensive, especially if you don’t hire many people. You get many cases at once. Data entry is still an issue. Weaknesses Challenges to generalizability—Hard to get a broad sample. Target population must be pretty specific. Scheduling. Requires a block of time convenient for all. Inaccuracy—behavioral problems in groups may distract respondents or even lead to “playful” responses Privacy—setting may allow others to view subjects’ responses Surveys—Group Administered

15. Surveys • Face-to-Face Interviews • Strengths versus Weaknesses

16. Strengths Permits complex questions—There’s someone to answer questions, explain, and probe. Highest response rate. Personal appeal increases likelihood that respondents will participate. Convenience—Allows rescheduling of visits (helps with response rate too). Reduces disability bias—interviewer may assist respondents CAPI: If using computer entry of responses, interviewing doubles as data entry. Weaknesses Cost. Payroll, driving, and so forth are expensive compared with other delivery methods. Interviewer Bias. The appearance and style of the interviewer can bias participation or answers. Sensitive topics may lead to hiding behavior from interviewer (e.g., drug use or sexuality). Differences between interviewers can lower reliability Oral format limits types of questions that you can ask, although visuals may be used. Interviewer safety is of concern e.g., fear can lead to falsified questionnaires. Surveys—Face-to-Face

17. Surveys • Telephone Interviews • Strengths versus Weaknesses

18. Strengths Permits complex questions There’s someone to answer questions, explain, and probe. Not as good as face-to-face. Higher response rate (maybe) Personal appeal increases likelihood that respondents will participate. Not as high as face-to-face. Convenience—Allows rescheduling of contact (helps with response rate too). Fast—surveys can be quickly completed Facilitates generalizability—Easy to reach more people over large area, cheaper than face-to-face but more expensive than mailed survey Environmentally friendly compared with other methods except electronic Weaknesses Cost. Payroll is still an issue, as is equipment, but is cheaper than face-to-face. Number of questions is limited by low tolerance for being on the phone for long. Biases: The linguistic style of the interviewer can bias participation or answers. Type of household that answers the phone. Who has a phone? Measurement Error Sensitive questions may le)ad to dishonesty (e.g., drug use or sexuality. If using area codes for sampling frame in an era of cell phones. Limits on the types of questions that you can ask: relies on short-term memory and quick recall visuals are not possible Falling response rates because of American attitude toward phone. Americans hang up on strangers or screen calls Phone interruption is an annoyance Surveys—Telephone

19. Surveys • Electronic Surveys • Strengths versus Weaknesses

20. Strengths Cheap. Depending on type, data are automatically entered by respondents Ease of editing questionnaire and analyzing data Fast—Distribution and response time is reduced. Reduced bias—You get more candid responses. One researcher found: Number of persons having sex with same-sex individuals doubles compared with mail and face-to-face surveys. Some complexity permitted. Interaction can be available, either through faqs, e-mail, or call to researchers. Environmentally friendly compared with other methods except telephone Can reach samples on a global scale Convenient for respondents Weaknesses Selection Bias & Generalizability: Must have computer access, but can be used to generalize to computer-access population. Lacks complete sampling frame Email contact may be treated as SPAM Construction of questionnaire can be challenging—often need web or computer expertise Challenges to accurate response: Assumes physical skills Literacy and language abilities Instructions may get tedious and confuse respondents Require electronic literacy This likely biases results. Glitches can kill participation for some Long-term response rates may be no better than other survey forms. Who fills it out? The Web Culture encourages “hacking” and falsification of information Surveys—Electronic

21. Surveys—Electronic • Of the types of electronic survey, those that carry us toward the population coverage of mail surveys and the response rates of the face-to-face survey will be judged the best. • Email/text/myface delivery with web page link or embedded app is likely to become the standard. The major obstacle is generating sampling frames for probability sampling of target populations.