COMM 301: Empirical Research in Communication

COMM 301:Empirical Research in Communication Lecture 8 Kwan M Lee

Survey / interview methodFunctions • Both survey and interview are self report techniques. • Able to serve two functions • Description • document current conditions • E.g., Opinion poll • Analysis • describe and explain why conditions occur • What are underlying causes and relationships?

Survey / interview methodAdvantages • high ecological isomorphism • Data gathered in real world situations • reasonable cost of data collection • access to broadly distributed population • Can cover broad location • access to subjective information • respondents’ private thoughts

Survey / interview methodDisadvantages • no control over independent variables • hard to rule out alternatives  IVs are not under control; they are simply measured • lack of temporal ordering • correlational research -- focus on relationships between variables • not causal relationships  because it cannot control time order • instrumentation bias • wording and organization of question items can affect the data collected – e.g., order effect • onus to develop questions well

Survey / interview methodDisadvantages (cont.) • requires respondents’ willingness • respondents may provide inaccurate or incomplete data • can be checked by survey instruments – e.g., parallel questions to check consistency • requires respondent recall • prestige bias • Respondents selected because of their specific SES may provide dishonest or biased answers • declining response rate • People hate it.

Survey/Interview Designs • Types (in terms of time span) • Cross-sectional vs. longitudinal • Cross-sectional design • data collected from a single point in time • create a description of the conditions at that time • equivalent to a still photograph, snapshot

Survey/Interview Designs (cont.) • longitudinal: data collected from a sample across a period of time • trend study • data collected from different samples selected from a single population at different time points • evaluate changes in the population • panel study • data collected from same sample at different time points • follow the sample over time to evaluate changes • examples: genetic predisposition to violence, television viewing and violence • vulnerable to which threat to validity? • attrition

Functions of the questionnaire • Designing survey instrument, aka the questionnaire • need to accomplish two tasks • convey information to respondent clearly and concisely, so that they understand what is being asked • provide a way for respondent to answer accurately and concisely

Question types • Question types, and their relative tradeoffs • closed ended • response options given by the question, respondent chooses the appropriate response • easier data input and analysis • lack detail • open ended • no predetermined response options, respondent can elaborate details • can collect data previously unknown to researcher • data analysis more problematic

Guidelines for developing questions • 1. Make questions clear • Meaning: no ambiguity • example of ambiguous, broad question: • “What do you think of our promotion?” • need to specify as much as appropriate • “what do think of the speed of rebate? ~ easiness of rebate instruction? ~ ~ ~ • simple, understandable language • avoid complex multi syllabi words, specialized terms (without clarification), and negative wording • Ninth-grade level • Avoid using words with multiple meanings • e.g. “hot”: Is your instructor hot?

Guidelines for developing questions • 2. Make questions concise • Good questions are short, yet clear • may consider breaking long questions into multiple questions

Guidelines for developing questions • 3. Include complete instructions • provide instructions about how to select items • rank order, choose only one, choose every items applicable? • clear instructions for filter questions • e.g. asking about email use for people without email • filter question: “Do you have access to email”  Yes  No (skip to Question 26)

Guidelines for developing questions • 4. Be realistic • About respondents’ abilities to recall • e.g. how many hours did you watch TV during the last year? ? ? • 5. Focus on purpose of survey • Control the length of a survey • Do not include items if they are not relevant to your research

Guidelines for developing questions • 6. Avoid double barreled questions • a single question that asks for more than one response • e.g. “Do you rely on the Internet to communicate with people and purchase merchandise?” • Use of conjunctions (and, or, but…) are yellow flags indicating possible double-barreled questions

Guidelines for developing questions • 7, 8. Avoid biased wording and leading questions • biased wording that can shape a response • e.g. “Do you read newspapers, or just watch television?” • other bias-prone words (words about ideology, political party…) • E.g., Do you agree with the Republican’s tax cut proposal? • leading questions that suggest specific responses • e.g. “Like most people, do you support anti-abortion? • e.g. “Don’t you support the anti-abortion proposals?” • double bind question • implies affirmative response regardless of how response • e.g. “Do you still cheat?”

Guidelines for developing questions(cont.) • 9. Avoid threatening questions • questions that cause unease or embarrassment • sensitive topics • sexual behavior, gambling, substance abuse, etc. • personal and private information • age, income,… • possible solutions • ask for equivalent answer if possible • e.g. year of birth instead of age • use categories, with an artificially low initial category • e.g. income • use random techniques • Number of cheating - A: 4; B:1; C:2; D:3

COMM 301: Empirical Research in Communication