240 likes | 464 Views
Effects of Survey Mode on Responses: An Experiment of Social Desirability Biases in Telephone , Interviewer , and Online Surveys. Indeok Song Joongbu University, Korea Sung Kyum Cho Chungnam National University. Contents. Background: 3 Surveys on Gambling Addiction
E N D
Effects of Survey Mode on Responses:An Experiment of Social Desirability Biases in Telephone, Interviewer, and Online Surveys Indeok Song Joongbu University, Korea Sung Kyum Cho Chungnam National University
Contents • Background: 3 Surveys on Gambling Addiction • Research Question & Purpose • Review of Previous Research • Methods • Results • Discussions & Suggestions
Research Question What is the reason for the inconsistencies among 3 surveys’ results? Which mode does produce the most accurate estimates? Sampling? Survey Mode?
Previous Research • Survey modes “can be equivalent but are not always identical” (Riva et al., 2003, p. 79) • Sensitivity & Social desirability (Booth-Kewley, Larson, & Miyoshi, 2007; Couper, 2000; Huang, 2006; Joinson, 1999; Kays, Gathercoal, & Buhrow, 2011; Tourangeau & Yan, 2007, 2011 and so on) • Mode of admin.: Self- vs. Interviewer-administered • Self-administration increases reporting of socially undesirable behaviors (Corkrey & Parkinson, 2002; Currivan et al., 2004; Hochstim, 1967; Moskowitz, 2004; Mott, 1985; Richman et al., 1999, and so on) ∴ Paper-Pencil, Online > Telephone, Interviewer
Previous Research • Interviewer or 3rd party presence • Social desirability bias is worse in telephone survey than in face-to-face interviews (Aquilino, 1994; Groves & Kahn, 1979; Holbrook et al., 2003; Johnson, Hougland, & Clayton, 1989; Leeuw & van derZouwen, 1988, and so on) • The presence of others affects responses to sensitive questions (Aquilino, Wright, & Supple, 2000; Harrison, 2001; Lombard & Ditton, 1997; Sudman, 2001; Tourangeau & Yan, 201, and so on) • Online survey yields the lowest level of other people’s presence ∴ Online> Interviewer> Telephone
Research Purpose Controlling for variances from sampling and different groups of samples, to examine the effects of survey modes on the responses to sensitive questions.
Methods • Online(n=21) • Pretest-posttest 3 between-group experimental design • 93 undergraduate students 66analyzed • Out of 66, male 38 (57.6%), female 28 (42.4%), mean age 20.61 (SD=1.97) • Paper-pencil(N=93) • 2 weeks • RandomAssignment • Telephone(n=24) • Interview(n=21)
Methods • Questionnaire Design • Addiction Scales • Gambling addiction (4pt., 9 items) • M=9.24, SD=2.54, α=.84 • Internet addiction (4pt., 15 items) • M=29.13, SD=7.24, α=.90 • Game addiction (4pt., 20 items) • M=26.80, SD=8.84, α=.96 • Dissatisfaction with self-appearance (5pt., 6 items) • M=14.43, SD=3.71, α=.66 • Sexual values (5pt., 10 items) • M=27.88, SD=7.14, α=.89
Methods • Sexualexperience • Age of the first experience • n=27, M=18.59, SD=1.67 • Number of sexual partners • n=26, M=10.81, SD=3.68 • Sexual behaviors (physical contact, cuddle, kiss, caress, intercourse) experienced in current relationship • n=19, M=3.68, SD=1.42 • Demographics • Gender, age, household-income
Results • Homogeneity among groups (pretest: paper-pencil)
2 x 3 Factorial ANOVAs • Independent Variables • 2 groups (‘Low’ & ‘High’) by median split • 3 modes (Online, Telephone, Interview) • Dependent Variables • Changes in responses = Pretest – Posttest
Changes in Responses for Dissatisfaction with Self-Appearance
Summary of the Results • Responses for ‘Game Addiction’ & ‘Dissatisfaction with Self-Appearance’ • Increased in the online survey mode • Decreased in the telephone & interview survey modes • Those who scored higher on the sensitive questions are more likely to change their responses in a socially desirable direction • Less addicted (gambling, Internet, game) • Less dissatisfied with self-appearance • Less liberal (more conservative) sexual values • Younger in the age of the 1st sexual intercourse
Discussions & Suggestions • Survey responses to sensitive (privacy-related) questions are affected by different survey modes. • Online survey in which respondents are less concerned about their privacies is more likely to elicit franker (more accurate) responses. • Paper-pencil (group administration), telephone, interview modes lead respondents to privacy concern & misreport. • Suggestion: Dual-mode survey Telephone Sampling Online Questionnaire