Week Three Readings: Context and Response Order Effects

1. Week Three Readings: Context and Response Order Effects BA531 Hongyan Shi and Adam Branson January 31, 2006

2. Cognitive Models: Answering Questions Processes, Context, & Response Order Effects On Attitude Measurement • SBS, Tourangeau et. al., & Krosnick et. al. • Survey • Survey Recap, Dissection, and Discussion • Additional Questions Related to Readings

3. Week Three Review • Chapter 4, 5, and 6 • Tourangeau and Rasinski (1988) • Krosnick and Alwin (1987)

4. Chapter Four and Tourangeau • Context Effects on Attitude Measurement at each stage of the Cognitive Model • Conditions at each stage in the response model • Variables • content, mood • reciprocity and norms • anchoring measurement, extremes, intervals, perspective and frequency effects • rank orders and rating scales • social desirability, self-presentation, consistency • survey format and mode

5. Chapter Five and Tourangeau • Substantive Information being brought to mind by preceding questions? • Inclusion/Exclusion Model • Assimilation and Carry Over • size of assimilation, expertise, # of preceding questions • Contrast and Backfire • sub-traction based and comparison based effects • What triggers these effects? Any variables that affect the categorization of information. • attitude stability and survey mode • Six Implications (Variables) for Questionnaire Construction • Preceding Content, # of Preceding Questions, Target Generality, Related Question Spacing, Introductions to Block or Non-Block of Questions, Questionnaire Layout • Attitudes: cognitive, affective, or conative or evaluative judgments • What are attitudes? Implications of File Drawer & Construal Model • Over Time, Strength, Accessibility (short-term or long-term memory)

6. Chapter Six and Krosnick • Order Effects Within A Question • primacy and recency effects • open response and forced choice (both large and small) • factual and judgment questions • Response Order Effect Causes • satisficing, memory limitations, option order, earlier item content • survey mode, time constraints, persuasive opinions, distractions • Cognitive Elaboration • serial position, presentation mode, and item plausibility • Judgmental Contrast Effects • dimensional and nondimensional, asymmetric, response continuums

7. Sample Survey • Overview and Instructions • How do you see material from chapters four, five, six and the outside readings incorporated into this survey? • How does this survey provide context effects? Assimilation? Contrast? Response Order? • What are particular items you could see affecting judgments of general audience pools?

8. Survey Questions • 1. People may associate foods with timely cultural, religious, and political festivals. Likewise, certain festivals may be associated with any numbers of foods. For example, people may celebrate New Years Eve by consuming Champagne or other sparkling wines. Can you think of any occasions associating consumable items and festivals? 9 Yes/ 0 No • 2. Which of the following foods do you most closely associate with Christmas? Please select one box. 2a. Response Options: 1 _Turkey 1 _Ham _Hamburgers 1 _Lobster 2b. Response Options: 1_Ice Cream _Pumpkin Pie 2 _Candy Cane 2 _Chocolate

9. Survey Questions • 3. Which of the following foods do you most closely associate with Thanksgiving? Please select one box. • 3a. Response Options _Turkey 3 _Ham _Hamburgers _Lobster • 3b. Response Options: 1 Ice Cream 4 Pumpkin Pie _Candy Cane Chocolate

10. Survey Questions • 4. Which of the following foods do you most closely associate with the Fourth of July? Please select one box. • 4a. Response Options _Turkey _Ham 3 _Hamburgers _Lobster • 4b. Response Options: 2 Ice Cream _Pumpkin Pie 3 Candy Cane Chocolate

11. Survey Questions • 5. Which of the following foods do you most closely associate with St. Valentine’s Day? Please select one box. • 5a. Response Options _Turkey _Ham 1 _Hamburgers 2 _Lobster • 5b. Response Options: _Ice Cream _Pumpkin Pie_Candy Cane 6 Chocolate

12. Survey Questions • 6. Please think of a food item from your own culture. Write, in the following space, the name of this food._apple pie, foie gras, doner, halwa, sweet bean soup, dumpling, pizza, ham stuffed bread, grits • 7. Is the food you wrote down in the prior question associated with a festival from your own culture? 4 Yes/ 5 No * If yes, please share the name of the festival._Christmas (2), New Year, Diwali

13. Survey Questions • 8. Is the food you wrote down in the previous question similar to the types of foods mentioned in this survey? I.e. Is it sweet? Is it a side-dish? Is it a meat dish? 5 Yes/ 3 No

14. Survey Questions • 9. On a five-point scale, please mark the corresponding box of how much you like this food. • 9a. Response Option One-Really Like Four-5 Two-4 3 Two-2 1 Really Dislike

15. Survey Questions • 10. On a five-point scale, please mark the corresponding box of how much you enjoy Chocolate Chip Cookies. • 10b. Response Option Really Dislike One-Moderately Dislike One-Neutral Three-Like Moderately Three-Really Like

16. Survey Questions • 11. Check one. In answering how much you enjoy Chocolate Chip Cookies, is the cookie number value based on a comparison with: __ Other Cookies _2 Other Desserts _1 Your Written Item _5 All Foods

17. Survey Dissection • Is Q1 general enough? Does the example serve a purpose? The repetitive word choice? What about the word choice for beverages, festivals, and “consumable”? • Are the different sets of Q2-Q5 useful? Should visual cues of the answer choices be provided? How would respondents react to a survey presented orally? Should response order options employ the “Latin Square”? What do you think about when you read the questions? Because these may not be salient events, do you “satisfice”? Were you lead by preceding questions? As you went on to the next question, did you want to go back and change any answers? Your attitudes?

18. Survey Dissection • Did Q6 lead you to believe that you should enter something that was primed? What would a respondent feel about going from closed questions to open response? • For Q6 and Q7, how would respondent and interviewer rapport influence the survey? What would be the various influences on the answer development processes? • Were you lead to thinking about a similar food category in Q8 as mentioned in Q2-Q5? Is this an indication of assimilation/carryover or did it indicate contrast/backfire effects?

19. Survey Dissection • In Q9, what was your basis for a comparison? Was your feeling and attitude about the item strong? Neutral? • Did it matter whether you began with really dislike and assigned a value of 1? Or, if you had a choice of really like and a value of 5? Should word options be the ranking scale or numbers? Should the range be broader? How did you anchor the item and compare it with something neutral, deplorable, favorable? • Did you compare it to items previously mentioned in the survey during questions Q2 – Q5?

20. Survey Dissection • Did having a switch from an item you wrote down to a seemingly innocuous item in Q10 create any anxiety? • Did you start guessing what might be next in the questionnaire? Were you comparing any festivals at this point to the target item? • Did having a reward for completing the survey bias your responses? • In Q11, were you comparing your rankings to other items not provided as a choice? Did having a closed set of options limit or force you to make comparisons you did not want to make?

21. Week Three • General Questions • Questions on Assumptions • Questions on the Inclusion/Exclusion Model

22. Chapter Four & Tourangeau General Questions • Do we agree human judgment is always context dependant? (SBS 81) • What would be necessary to answer differently? • What problems do you see with the model assumptions? • What can researchers be doing to ensure intentional priming materials are used by respondents? Does it matter? • If individuals seek consistency during surveys, what does this imply for group or organizational level surveys? Will we see a normal probability distribution function? Or, is it group consistent? • Are there occasions when greater familiarity or expertise with subject matter does not result in greater consistency over time? When are we not prone to context effects? • What are the impacts of “textbook” and “sensory experienced” knowledge on attitudes and judgments?

23. Chapter Five General Questions • Why are context effects not stable or replicable? What does this mean for the surveying discipline? • What explains when preceding items do not result in assimilation or contrast effects? Can we determine this? • What reasoning best explains why expertise results in mixed results of context effects? • What are respondents answering when they provide information on their attitudes? • Does the SBS pg. 109 flow chart present a complete portrayal of how effects emerge? • What are the strengths and weakness of the construal model? • What other variables may elicit inclusion and exclusion? • How can we isolate these context effects?

24. Inclusion/Exclusion Model Assumptions • Dynamic nature of knowledge representation. Or, structures in long term memory? • Representation of target stimulus and the representation of the standard are, in part, context-dependent…. ? • Respondents who are highly knowledgeable about an issue should be less susceptible to context effects than those who are less knowledgeable…. ?

25. Chapter Six & Krosnick General Questions • Do we agree it is possible to predict response order effects? Is it possible to mitigate or isolate response order effects? • How would primacy or recency effects appear in open-ended questions? • Do primacy effects arise only because of respondent personality and their interaction with the survey instrument? • Is it more likely that primacy and recency effects can cancel each other out across large population samples? Or, will we have distortions? • Given this information, why is it or is it not possible to predict and mitigate context effects? • What would we see in the Krosnick study if the examination was based on an aural/oral survey? • What is the value in increasing respondent motivation to consider all possible options? Is there a trade-off? Do we initiate other context effects?

26. SBS Ch. Six Assumptions and Questions • Memory limitations are not the primary source of response order effects? • Limitations are more likely to result in recency effects for complex questions? • Should we consider the complexity of each alternative? 134, 248, 100, 752, 553 • Interaction of serial position, presentation mode, and plausibility/persuasiveness? • Do you concur response order effects are less pronounced when a formed judgment can be recalled from memory?

27. Other Topics/Interesting Items • What are your other thoughts and comments regarding the improper and proper use of techniques employed in this survey and those techniques discussed in the readings for this week?

28. Week Three-Conclusion • Cognitive Models: Answering Questions Processes, Context, & Response Order Effects on Judgments and Attitudes • SBS, Tourangeau et. al., & Krosnick et. al. • Additional Questions Related to Readings • Survey • Survey Recap, Dissection, and Discussion • On To Autobiographical Memory

29. Thank you!