1 / 46

Question Wording

Question Wording. Class 25. ANNOUNCEMENTS. OPTIONAL REVIEW SESSION: THURS, 12/12 STATS EXERCISE DUE THURS, 12/12 FINAL : THURSDAY, 12/19 – SMALL CONF ROOM, SMITH 302 11:30 – 2:30 FORMAT: MULTIPLE CHOICE, SHORT ANSWER COVERS: FROM MIDTERM THROUGH TODAY

lowell
Download Presentation

Question Wording

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Question Wording Class 25

  2. ANNOUNCEMENTS • OPTIONAL REVIEW SESSION: THURS, 12/12 • STATS EXERCISE DUE THURS, 12/12 • FINAL: THURSDAY, 12/19 – SMALL CONF ROOM, SMITH 302 • 11:30 – 2:30 • FORMAT: MULTIPLE CHOICE, SHORT ANSWER • COVERS: FROM MIDTERM THROUGH TODAY • COURSE EVALUATIONS – SIRS SURVEYS http://sirs.rutgers.edu/

  3. Road Bumps in Human Communication (Watzlawick, Beavin, & Jackson, 1967) Watzlawick (Vaht-slahv-ick) applied for an assistantship with a psychiatric research institute.  The following conversation took place with the receptionist. WATZ:  Good afternoon, I have an appointment with Dr. H. My name is Watzlawick RECEPTIONIST:  I didn't say it was. WATZ.:  (taken aback and somewhat annoyed).  But I'm telling you it is. RECEPTIONIST:(bewildered) Why then did you say it wasn't? WATZ.:  But I said it was. WATZ:  Why am I the object of some opaque but disrespectful joke? RECEPTIONIST:  First this guy says “my name is not Slavic”, then he says it is Slavic. He must be a new psychotic patient of Dr. H's.

  4. Road Bumps in Human Communication (Watzlawick, Beavin, & Jackson, 1967) Watzlawick (Vaht-slahv-ick) applied for an assistantship with a psychiatric research institute.  The following conversation took place with the receptionist. WATZ:  Good afternoon, I have an appointment with Dr. H. My name is not Slavic. RECEPTIONIST:  I didn't say it was. WATZ.:  (taken aback and somewhat annoyed).  But I'm telling you it is. RECEPTIONIST:(bewildered) Why then did you say it wasn't? WATZ.:  But I said it was.

  5. Open Ended and Closed Ended Questions Open Ended: Where respondent is free to provide an answer in his or her own words. "What topics lead to arguments in your family?" Mummy insists that we wear academy blazers and ties to Sunday dinner, but Natasha and Sterling just want to wear their day-glo capes and studded collars. Closed Ended: Where the respondent is restricted to a pre-set range of response options. "What topics lead to arguments in your family?" ___ Finances ___ Noise _X_ Decorum (manners, clothing, make-up) ___ Household chores ___ Opinions about social issues ___ Other

  6. Closed Ended vs. Open Ended Questions Closed Ended Defines question more clearly Defines response options more clearly Does not require an interviewer Easy for respondent Easy for coding/analysis Easy for data entry Open Ended Provides for unexpected responses Less likely to bias responses Enhances respondent good-will Historical record

  7. Not at all Slightly Somewhat Quite a Bit To a Great Degree Poor 1 2 Fair Good 3 Very Good 4 Excellent 5 1 2 3 4 5 Subjective Response Scales How would you rate the competence of your court-appointed family mediator? To what degree are family conflicts resolved in a way you find satisfactory?

  8. Considerations Regarding Questionnaire Item Format 1. Limited interpretability of subjective response scales 2. Agree-disagree options 3. "Don't Know" option 4. Middle position choice options 5. Unbalanced vs. balanced format 6. Measures of attitude strength

  9. Good Fair Poor How Do You Rate Robeson Cafeteria Food?

  10. Superb Very Good Good Fair Poor How Do You Rate Robeson Cafeteria Food?

  11. Agree/Disagree Format Agree Disagree • I wish my family would be more courteous. • I am tired of having to watch what I say. • Venting pent up feelings is selfish. • My family needs to see a conflict mediator. Problems with this format?

  12. Problems With Agree/Disagree Format • Lose info when squeezing range of attitudes into a dichotomy. Can’t correlate responses w’ each other. • Works best for extreme attitudes. Can’t distinguish moderate or ambivalent att’s from extreme att’s. • Confusing to indicate a negative (or positive) attitude by responding in the affirmative (or negative). Fowler discourages use of agree/disagree for these reasons.

  13. What Difference Does It Make To Include / Exclude DK (“Don’t Know”) Response? 1. Does it change rate at which people say DK? 2. Does it change the rate at which substantive (non-DK) responses are endorsed, relative to one another? 3. Does DK option change relationships between variables?

  14. Response Type and Rate as a Function of DK Filter

  15. Solution: Provide DK option.

  16. “Floaters”: A Problem and a Mystery Floaters: People who give a response when there is not a DK (“don’t know”) filter, but who reply “Don’t Know” when there is a DK filter. Relations between responses changes when DK filters are used. * Floaters weaken true correlations * Floaters can create false correls, where none actually exist * Floaters sometimes make no difference at all Who are floaters? Difficult to say. For obscure questions, floaters tend to be among the less educated.

  17. Schwartz: Self Reports 1. How people make sense of questions: Pragmatics of Communication 2. How people answer behavioral questions Cognitive strategies and heuristics 3. Special case of attitude questions: Context effects

  18. Literal Meaning and Pragmatic Meaning • Literal Meaning: • How would you articulate the chronologically proximal condition of your • limbic-related subjective state? • Q. How would you describe your current mood? • Pragmatic Meaning: • Q. How would you describe your current mood? • (Do they want to know what I'm feeling right now?) • (Do they want to know how I'd express my feelings, e.g. in words, in song, through interpretive dance?)

  19. Grician Maxims and the • Pragmatics of Communications • 1.Relation: Contribution must be relevant to conversation. • Quantity: Provide sufficient information: • not too little, not too much. • 3.Manner: Information should be relayed clearly and succinctly, not ambiguous and not over-wordy. • 4.Quality: Information should be true, not false.

  20. Inquiry Maxim Invoked Violation of Maxim Adherence to Maxim What brings you here today? Relation My Car My sinuses What is the nature of your problem? Quantity I feel bad. [OR] My sinuses, located throughout my skull and producing mucus, I think I may have a sinus infection. What are your symptoms? Manner As the Bard mused upon the stoppage of the River Thames… My nose is all stuffed up. Have you taken meds I prescribed Quality Sure have—every day [a lie]. I keep forgetting—sorry [the truth] Grician Maxims Go to the Doctor

  21. Whoa! Way too much information!

  22. How do I get to Kent? Go to Smith Hall, third floor, Room 352  Tell him Social Psychology sucks

  23. HEADLINES THAT VIOLATE GRICIAN MAXIMS

  24. Relevance of Grician Maxims To Survey Questions 1. In normal conversation, people assume others are following these maxims. 2. In surveys, people make the same assumption: a. All info. in surveys serves to convey main points 1) Wording 2) Cues in the format, structure, order 3. People use maxims to figure out how to answer survey questions.

  25. Closed-Ended Questions Help Clarify Meaning of Inquiry “What is the most important thing for children to prepare them for life?” _______________________________________ OPEN ENDED “What is the most important thing for children to prepare them for life?” A. _____ Learning math CLOSED ENDED B. _____ Language skills C. _____ Politeness and manners D. _____ To think for themselves E. _____ Experimental methods “To think for themselves” Open ended endorse this: 04.6% Closed ended endorse this: 61.5% Note: The effect of closed-ended questions is probably a combination of added clarity plus suggesting responses that do not spontaneously occur to respondent.

  26. "Frequent" Frame "Infrequent" Frame Once a day Once a week 4-6 times a week 2-3 times a month 2-3 times a week Once a month Once a week Once every 2-3 months Once a month Once every 6 months Frequency Scales and Question Interpretation Q. How often do conflicts arise in your family? x x

  27. "Frequent" Frame "Infrequent" Frame Once a day Once a week 4-6 times a week 2-3 times a month 2-3 times a week X Once a month X Once a week Once every 2-3 months Once a month Once every 6 months Frequency Scales and Question Interpretation Q. How often do conflicts arise in your family? Frequent Common events Infrequent  Extreme events

  28. Framing Past Events Affects Reports of Current EventsWinkielman, Knaüper, & Schwarz, 1998 Q1: “How often do you get angry during a typical (week/year)? Hardly Ever Very Frequently 1 2 3 4 5 6 7 8 9 Q2: Based on Q1how would you answer the following (1-10 scale)? “Week” “Year” Intensity of feelings around events 5.74 6.37 Seriousness of events 5.32 6.37 Answer to Q1 (“How often?”) 3.77 4.26

  29. Not at all Not at all Barely at all Barely at all Slightly Slightly Some-what Some-what Moder-ately Moder-ately Very Very Completely Completely -3 0 1 -2 2 -1 0 3 4 1 2 5 3 6 Rating Scale Format as Cue to Question Meaning Q. How satisfied are you with the way family members respond to your concerns? Q. How satisfied are you with the way family members respond to your concerns?

  30. Not at all Not at all Barely at all Barely at all Slightly Slightly Some-what Some-what Moder-ately Moder-ately Very Very Completely Completely -3 0 1 -2 -1 2 3 0 1 4 2 5 3 6 Rating Scale Format as Cue to Question Meaning Q. How satisfied are you with the way family members respond to your concerns? Q. How satisfied are you with the way family members respond to your concerns? Note: Bipolar implies neg & pos, unipolar implies pos only

  31. Researcher Affiliation as a Context Cue "Hi. I'm from the Institute for Personality Research. Why do you think family conflicts occur?" * Hostile types in this family * People lack communication skills "Hi. I'm from the Institute for Social Research. Why do you think family conflicts occur?" * No time for family dinners * Economic pressures

  32. Inferring Meaning of Questions from Adjacent Questions Target Question: Do you support the newly instituted educational contribution?[IN FACT NO SUCH POLICY EXISTS] Preceding question A. Do you favor the American system where students pay their own tuition? B. Do you favor the Scandinavian system of government sponsored college education? Students favored contribution more when preceded by Question A or B?

  33. Inferring Meaning of Questions from Adjacent Questions Target Question: Do you support the newly instituted educational contribution? [IN FACT THERE IS NO SUCH POLICY] Preceding question A. Do you favor the American system where students pay their own tuition? Implies that contribution will come from students. B. Do you favor the Scandinavian system of government sponsored college education? Implies that contribution will come from government. Students favored contribution more when preceded by Question B than by Question A.

  34. Psychological Influences on Question Response How often do people in your family have conflicts? I. Cognitive tasks: 1. Recall of information 2. Computation of judgment 3. Format judgment in terms of choices II. Motivational/emotional issues 1. Self presentation 2. Social desirability

  35. ResponseFormat as Memory Cue

  36. ResponseFormat as Memory Cue

  37. Number of Response Options Imply Norms

  38. Number of Response Options Imply Norms

  39. Survey Format Affects Interpretation of Responses • People who review survey results will draw format-based inferences about the meaning of responses. • Example: MD estimates of illness severity shaped by patient reports, which are shaped by survey response options.

  40. Effects of Outcome (e.g., Symptom) Clarity and Frequency on Outcome Reporting Vague Symptom (e.g., "feeling unwell") Clear Symptom (e.g., "fever")

  41. How Later Questions Can Influence Responses to Earlier Questions

More Related