Survey Research. Sampling and Inference June 9, 2008

1. Survey Research. Sampling and InferenceJune 9, 2008 Ivan Katchanovski, Ph.D. POL 242Y-Y

2. Survey Research • Surveys: One of the main methods of data collection and research in political science and sociology • Polls: Used in mass media and business • Advantages • Reliable • Involves relatively small numbers of respondents • Relatively cheap and fast • Compare with census • Disadvantages • Sampling error • Validity

3. Example of Polls: Bush Approval

4. Observation and Sampling • Polls and surveys are based on observations • Population: universe of subjects the researcher wants to sample • Adult population in Canada • Voters in Ontario • Sample: cases or observations drawn from a population

5. Random Sample • Random selection: Each element has an equal chance of selection independent of any other event in the selection process • Are internet polls random? • Are telephone polls random? • Weighting: Giving some cases in a survey dataset more weight than others • Weights examples

6. Parameters and Sample Statistics • Parameters: Characteristics of a population • Proportion of the vote for political parties in national or local elections • Median personal income in Canada • Sample Statistics: Estimates of population parameters, based on a sample drawn from a population • Exit poll estimate of proportion of the vote for political parties in national or local elections • Survey estimate of median personal income in Canada • Inference about the characteristics (parameters) of a population based on samples

7. Confidence Level • Surveys and polls: margin of error • Random sampling error: • Decreases with increase in sample size • Confidence Level • The estimated probability that a population parameter lies within a given confidence interval • Example: we might be 95% confident that between 48 and 52% of all voters favor Candidate A.

8. Election Eve Polls - U.S. Presidential Candidates, 2004

9. Types of Survey Questions • Open-ended questions • Respondent is asked to provide his or her own answer to the question • Closed-ended questions • Respondent is asked to select an answer from among a list provided by the researcher • Example: different % correct responses in open-ended and closed-ended questions

10. Example: Two Types of Questions in a 2007 Pew Poll of Americans Open-Ended Question % Closed-Ended Question, % Who is the President of Russia? Is it… (A) Vladimir Putin 60 (B) Boris Yeltsin 7 (C) Mikhail Gorbachev 7 (D) or is it someone else 8 Don’t know (Vol.) 18 Total 100 Can you tell me the name of the president of Russia? Yes, Vladimir Putin 36 Yes, other incorrect 3 No, don’t know 61 Total, 100

11. Contingency Questions and Biased Questions • Contingency questions: Questions which are contingent on response to previous questions • Example of a contingency question: Q14.1a. When it comes to political matters, do you ever think of yourself in terms of Left and Right? (1) Yes (2) No (9) DK, etc. (IF YES TO Q.14.1a, ASK Q.14.1b) 14.1b) How would you place your views on this scale, generally speaking – where 1 means “very Left” and 10 means “very Right”? • Biased questions: Questions that encourage respondents to answer in a particular way

12. Response Rate • Number of people participating in a survey divided by the number selected in the sample • Acceptable response rates • 50% - adequate for analysis and reporting • 60% - good • 70% - very good • Lower response rates • Can be used if non-respondents are similar to respondents