Effective Methods in Survey Research for Mass Communication Studies

1. Study on Method of Mass Communication Research传播研究方法 （9&11）Dr. Yi Mou 牟怡

2. Survey

3. An early example • Does your employer or his representative resort to (诉诸) trickery in order to defraud you of a part of your earnings? • If you are paid piece rates, is the quality of the article made a pretext for fraudulent deductions from your wages?” -- A survey by Karl Marx (25,000 questionnaires were mailed out to French workers, without any record of any being returned).

4. Topics Appropriate for Survey Research • Descriptive, exploratory, and explanatory • Units of analysis = respondents • Respondents – A person who provides data for analysis by responding to a survey questionnaire. • Large samples, original data, measuring attitudes and orientations

5. Guidelines for Asking Questions Survey (调查) ≠ Questionnaire (问卷) Questionnaire – A document containing questions and other types of items designed to solicit information appropriate for analysis.

6. Choose Appropriate Question Forms • Questions and Statements • Questions: • Statements: • Scales

7. Choose Appropriate Question Forms • Open-Ended and Closed-Ended Questions • Open-Ended Questions – Questions for which the respondent is asked to provide his/her own answers. • Closed-Ended Questions – Survey questions in which the respondent is asked to select an answer from among a list provided by the researcher. • Categories should be exhaustive (穷尽的). • Categories should be mutually exclusive (排他的).

8. Principle • Make Items Clear • 在过去的一段时间里你的社交媒体使用情况怎样？ • 在过去的24小时里，你使用微博的时间大概是___分钟？

9. Principle • Avoid Double-Barreled Questions • 我相信新浪微博上的消息的真实性和权威性。 Strongly Disagree – Strongly Agree

10. Principle • Respondents Must Be Competent to Answer • Consider the respondents’ literacy level

11. Principle Avoid Biased Items and Terms

12. Principle • Questions Should Be Relevant • Each item must measure a variable!! • Respondents Must Be Willing to Answer

13. Principle • Short Items are Best • Karl Max’s BAD examples.

14. Principle • Avoid Negative Items • “对我来说我觉得向那些与我意见不同的人表达我的观点很容易”。 • Strongly Disagree – Strongly Agree

15. Additional Principles • Avoid leading questions • “你认为。。。吗？” • Asking questions on variables, not relationships • “你认为A和B有关系吗？”

16. Questionnaire Structure • General Questionnaire Format • Uncluttered • One question per line • Consistent format

17. Formats for Respondents

18. Contingency Question 关联问题– A survey question intended for only some respondents, determined by their responses to some other question. • “逻辑问题”

21. Matrix Questions

22. Ordering Items in a Questionnaire • Appearance • Open-Ended or Closed-Ended First? • Randomized Ordering • Sensitivity to the Problem • Demographic questions should go at the end • Really?!

23. Questionnaire Instructions • Introductory comments and clear instructions • Pre-testing the Questionnaire

27. Self-Administered Questionnaires Questionnaires in which respondents are asked to complete the questionnaire by themselves.

28. Online Surveys • Procedure: • Invitation • Anonymity • Collecting data • End survey • Representative?

29. DO use consistent wording. • DO use simple language. • DON’T force excessive scrolling. • DO offer to share select result with respondents. • DO plan time and day of initial mailing. • DO be aware of technical limitations. • DO test incentives, rewards, and prizes. • DO limit studies to less than 15 minutes.

30. Mail Distribution and Return • Why do people not return questionnaires? • Monitoring Returns • Follow-Up Mailings

31. Response Rate – The number of people participating in a survey divided by the number selected in the sample. • Ideal = higher than 70% • Why is a low response rate bad? • What can be done to improve response?

32. Interview Surveys Interview – A data-collection encounter in which one person (interviewer) asks questions of another (respondent).

33. The Role of the Survey Interviewer • Interviewers solicit higher response rates (80-85%) than mail surveys. • Interviews minimized “don’t know” and “no answer.” • Interviewers serve as a guard against confusion. • Interviewers can observe respondents while completing the questionnaire.

34. General Guidelines for Survey Interviewing • Dress appropriately • Be familiar with questionnaire • Follow question working exactly • Record responses exactly • Probe when necessary • Probe – a technique employed interviewing to solicit a more complete answer to a question.

35. Coordination and Control • Training • General guidelines • How to handle difficult situations • Practice interviews • “Real” interviews

36. Telephone Surveys • Advantages • 95.5% of households have a telephone？ • Time and money • Control • Personal safety • Disadvantages • Bogus surveys • Unlisted phone numbers • Cell phones • Answering machines/voicemail/caller ID

37. Random-Digit Dialing (RDD) – A sampling technique in which random numbers are selected from within the ranges of numbers assigned to active telephones.

38. Computer-Assisted Telephone Interviewing (CATI) – A data-collection technique in which a telephone-survey questionnaire is stored in a computer, permitting the interviewer to read the questions from the monitor and enter the answers on the computer keyboard

39. Comparison of the Different Survey Methods • Self-Administered Questionnaires • Cheaper and faster than face-to-face interviews • National is the same cost as local mailings • Requires small staff • More willingness to answer controversial items • Interview Surveys • Fewer incomplete questionnaires • More effective for complicated questionnaires • Face-to-face is more intimate • Telephone Surveys • Cheaper and more time efficient • Online Surveys • Available software and websites

40. Strengths and Weaknesses of Survey Research • Strengths • Useful in describing large populations • Make large samples possible • Surveys are flexible • Standardized questions • Weaknesses • Round pegs in square holes • Seldom deal with context of social life • Inflexible • Artificial • Weak on validity (but strong on reliability)