1 / 18

Non-sampling Errors in Polls and Surveys

Non-sampling Errors in Polls and Surveys. Dru Rose . Surveys and Polls. Observational Studies. Experimental Studies. Methods of Data Collection. Dru Rose . Polls and Surveys. Poll. Survey. Many questions May have branches and skips May have a “tick multiple boxes” option

dooley
Download Presentation

Non-sampling Errors in Polls and Surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Non-sampling Errors in Polls and Surveys Dru Rose

  2. Surveys and Polls Observational Studies Experimental Studies Methods of Data Collection Dru Rose

  3. Polls and Surveys Poll Survey Many questions May have branches and skips May have a “tick multiple boxes” option May have open-ended, write a comment questions • Few questions • Multi-choice type only Dru Rose

  4. Sampling Errors (random process) Non-sampling Errors Sampling Selection bias Non-response bias Self selection bias Question effects Behavioural considerations Interviewer effects Survey-format effects Transfer of findings

  5. Target population (e.g. adults in NZ) Selection Bias: Population sampled is not exactly the population of interest. Sampling frame (e.g. households with a landline phone) Not included in sampling frame Cannot be contacted SAMPLED POPULATION Not eligible for survey Refuse to respond Incapable of responding

  6. Sources of Non-sampling Errors Non-response bias When people who have been targeted to be surveyed do not respond: Potentialbias if non-respondents are likley to behave differently to respondents with respect to the question being asked. e.g. Non-respondents in an employment survey are likely to be those who work long hours.

  7. Sources of Non-sampling Errors Self-selection bias People decide themselves whether to be surveyed or not. Dru Rose

  8. Self-selection bias: phone-in or internet polls

  9. Sources of Non-sampling Errors Question effects Subtle variations in wording can have an effect on responses. e.g. “Should euthanasia be legal?” vs. “Should voluntary euthanasia be legal?” People are more likely to favour “voluntary” euthanasia.

  10. 18 August 1980 New York Times/CBS News Poll “Do you think there should be an amendment to the constitution prohibiting abortions?” Yes 29% No 62% Later the same people were asked: “Do you think there should be an amendment to the constitution protecting the life of the unborn child?” Yes 50% No 39%

  11. Ethnicity 1986 1991 1996 94.6 94.3 Single 81.0 Ethnicity 81.2 78.1 European 65.8 9.1 9.6 Maori 7.6 Two Ethnicities 4.0 4.5 11.2 2.9 2.7 European & Maori 4.7 0.0 0.6 Two European gps 4.5 Question Effects in the NZ Census 1986: “What is your ethnic origin? (Tick the box or boxes which apply to you.) 1991: “Which ethnic group do you belong to?” (Tick the box or boxes which apply to you.) 1996: “Tick as many circles as you need to show which ethnic group(s) you belong to.”

  12. Sources of Non-sampling Errors Behavioural considerations People tend to answer questions in a way they consider to be socially desirable. e.g. pregnant women being asked about their drinking habits maybe reluctant to admit that they drink alcohol Dru Rose

  13. Sources of Non-sampling Errors Interviewer effects Different interviewers asking the same question can obtain different results. e.g. the sex, race, religion , manner of the interviewer may influence how people respond to a particular question. Dru Rose

  14. Interviewer Effects in Racial Questions In 1968, one year after a major racial disturbance in Detroit, a sample of black residents were asked: “Do you personally feel that you trust most white people, some white people or none at all?” • White interviewer: 35% answered “most” • Black interviewer: 7% answered “most”

  15. Sources of Non-sampling Errors Survey-format effects -question order e.g. “To what extent do you think teenagers are affected by peer pressure when drinking alcohol ?” followed by: “ Name the top 5 peer pressures you think teenagers face today.” -survey layout -interviewed by phone or in-person or mail.

  16. Auckland sample New Zealand Sources of Non-sampling Errors Transferring findings Taking the data from one population and transferring the results to another. e.g. Auckland opinions may not be a good indication of New Zealand opinions.

  17. Non-sampling Errors • can be much larger than sampling errors • are always present • can be virtually impossible to correct for after the completion of survey • virtually impossible to determine how badly they will affect the result • good surveys try to minimize them in the design of the survey (e.g. do a pilot survey first)

  18. Surveys/Polls A report on a sample survey/poll should include: • Who carried it out and who funded it • target population (population of interest) • sample selection method • the sample size and the margin of error • the date of the survey • the exact question(s) being asked • the results • the claims (inferences) made

More Related