180 likes | 345 Views
Designing and Using a Survey. February 7, 2011. Objectives. By the end of this meeting, participants should be able to: Explain how surveys accomplish the goals of: measuring attitudes, measuring change over time, making group comparisons, and analyzing the causes of behavior.
E N D
Designing and Using a Survey February 7, 2011
Objectives By the end of this meeting, participants should be able to: Explain how surveys accomplish the goals of: measuring attitudes, measuring change over time, making group comparisons, and analyzing the causes of behavior.
Goals of Surveys • Measure the frequency of attitudes, beliefs and behaviors • Measure change over time • Examine differences between groups (race, class, gender, etc.) • Analyze causes of attitudes, beliefs, and behaviors
Measuring Behavior • Time can be awkward for respondents • Forward telescoping • Backward telescoping • Sensitive subjects • Bogus pipeline technique (ethical concerns?) • Random lists • Randomized response technique (some sensitive, some not) • Importance of others’ beliefs
Predicting Behavior • Election Predictions • Problems with third parties • Problems with really close elections • Problem with social desirability for turnout • Undecided • Sensitive topics
2008 Presidential Example- NC Final Results Obama 2,142,651 49.70% McCain 2,128,474 49.38% Others 39,726 0.92% Total 4,310,851 Source: Office of the Clerk of the US House of Representatives
How did the individual polls do? FOX News/Rasmussen McCain MofE Reuters/Zogby McCain MofE American Research Group Obama MofE SurveyUSA McCain MofE Mason-Dixon McCain MofE Research 2000 Obama MofE Politico/InsiderAdvantage Tie MofE
Recall: Margin of Error • So for example, FOX/Rasmussen’s final poll for North Carolina showed 50% for McCain and 49% for Obama • Sample: 1000 likely voters • Margin of error = ±t {p(1−p)/(n−1)} 1/2 {1 − f}1/2 • But since the population being sampled is large we can ignore the final {1 − f}1/2 • ±1.96 {50(50)/(1000-1 )} 1/2 • = ±3.1% • In other words, Rasmussen/FOX was 95% confident that the final vote for McCain would be between 46.9% and 53.1%. Right on the mark, pretty much. • Yet, if they had to make a prediction of the winner, they’d guess wrong. (Some polls had MO wrong, too.)
Measuring Attitudes • One of the most common goals of surveys • Need to be concerned about measuring non-attitudes (Converse) • Need to consider attitude strength, respondents may not care equally about all issues (environment, gun control, etc.) • How do they feel about the issue personally? • How knowledgeable are they about this issue? • How certain are they of their opinion? • How much thought they have given to the issue? • Should you offer counterarguments?
Measuring attitudes • Attitudes especially weakly held attitudes will change over time (first impression versus considered opinions) • Need to watch disagreements over the meaning of words • Example: ideology and Stimson’s work • Need to watch for changing frames • Verbal (example: welfare, next page) • Contextual (example: sexual harassment)
Measuring Change over Time • Measurement of personal attitude change is frequently unreliable • Comparisons of cross sectional surveys are more common but pose their own issues • Similar interviewing methods? • Similar sample sizes? • Similar question wordings? • Does the change exceed sampling variation?
Measuring Change over Time • Frequently can be atheoretical • Can only measure gross change, not change at the individual level • Insta-polls can allow for immediate reactions to stimuli but suffers worries about representativeness and long term impact
Measuring Change over Time • Preferred method: panel studies • Repeated studies of the same individuals over time • Limitations • Long term studies are prohibitively expensive • Atrophy of the original sample (and worries about those that remain) • Errors in interviewing
Sub-group Comparisons • Researchers frequently want to compare the attitudes of various sub-groups of the population • One difficulty is the smaller and less accessible groups may be harder to randomly sample • Double sample or over sampling • Pyramiding or combining multiple surveys
What Leads to Change in Attitudes over Time? • Surveys generally struggle with this type of question • Most people’s answers for why their opinions changed are post-hoc rationalizations • Individuals may not be sure of their reasons for opinion change • Similar stimulus may affect people differently
For February 9: Work on research proposals, you will have time for group work next meeting. • For February 11: Download the program R from http://www.r-project.org/ and bring your laptop to class. You also may want to print-out “Getting Our Feet Wet with R” from the website. • For February 14: • Research proposals due for each group. • Read WKB Chapter 8 • On p. 189 answer either question 1, 2, or 3. • Construct a causal model relevant for YOUR research question. (Turn-in the C.8 items individually.)