1 / 40

Survey Design Issues

Survey Design Issues. Sampling. Here are some issues relating to sampling…. What sampling procedure are they using?. What sampling procedure is he using?. Response Rate Examples. Here are some examples of increasing response rate…. How are they increasing response rate?.

Mercy
Download Presentation

Survey Design Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Survey Design Issues

  2. Sampling • Here are some issues relating to sampling…

  3. What sampling procedure are they using?

  4. What sampling procedure is he using?

  5. Response Rate Examples • Here are some examples of increasing response rate…

  6. How are they increasing response rate?

  7. How are they increasing response rate?

  8. Question Examples • Here are some sample questions from a variety of surveys, with my thoughts about them…

  9. Never?

  10. Enjoy?

  11. What are they measuring?

  12. What about these questions?

  13. You have to offer every option, no matter how odd

  14. Not applicable?

  15. Response choices mismatched…

  16. Not applicable?

  17. Why am I answering questions about a previous provider when I might not need to? This question offers the option, but they should be bundled under one filter question.

  18. Not applicable?

  19. It’s not good to force a response when a person doesn’t know what the thing is.

  20. It’s not good to force a response when a person isn’t what the thing is.

  21. It’s not good to force a response when it isn’t relevant. Filter…

  22. I said “no.” Filter…

  23. I said “no.” Filter…

  24. General to specific

  25. A Poll Example • This poll was posted on CNN. I went through the comments to highlight various issues with survey design.

  26. A Poll Example CNN Poll: Obama approval rating back to 50% mark http://politicalticker.blogs.cnn.com/2012/02/15/cnn-poll-obama-approval-rating-back-to-50-mark/ Washington (CNN) - President Barack Obama's approval rating is back to 50% for the first time in more than eight months, and he currently holds an edge against all the remaining Republican presidential candidates in hypothetical head-to-head match-ups, according to a new national survey. And a CNN/ORC International Poll released Wednesday also indicates that the GOP's advantage on enthusiasm has been erased, and that the number of Americans who think things are going well in the country is on the rise. Six out of ten say things are going poorly in the country, but four out of ten say things are going well, up 15 points since November.

  27. Comments Klkl says: (SAMPLING) February 15, 2012 04:10 pm at 4:10 pm I always laugh when cnn do everything they can to make the president look good. Also that 1,026 are the majority of people in the world according to polls. Any kind of poll survey makes me laugh at see this is good estimate of how all americans feel by getting a limit amount of people opinions and see this is how everybody else feels about the subject.

  28. Comments REPUBLICAN ARE THE GREATEST says: (SAMPLING) February 15, 2012 04:16 pm at 4:16 pm They must have polled the uneducated people in America. Anybody who votes for Obama needs their heads examined,

  29. Comments Truth-Bomb Thrower says: (QUESTIONS) February 15, 2012 04:33 pm at 4:33 pm And the Obama media is kicking their re-election efforts into high-gear. A crafty pollster could probably also come up with a poll that says 50% of americans believe in Bigfoot. It’s all in the wording. But it doesn’t matter. Americans are obedient little sheep. They’ll believe whatever the media tells them to. What did people used to do before there were pollsters and pundits to tell them what to think?

  30. Comments Truth-Bomb Thrower says: (QUESTIONS) 30% DO believe in bigfoot.

  31. Comments Jharris says: (VALIDITY) February 15, 2012 04:08 pm at 4:08 pm Well, there is no science to these polls for Obama. If the economy is doing better, people view the president better. If the economy worsens, people view the president in the negative view. There is a lot of division in the GOP which helps the president, but there a long time until November and really these polls don’t mean squat because it is just a snap shot.

  32. What about this point? Are numbers enough?

  33. Note the struggle: All or none responses and no way to clarify an answer.

  34. Quantitative vs. Qualitative • A survey was conducted to evaluate researcher practices in social psychology research. Here is one question: • How often do you falsify data? • (This research was conducted by Matt Motyl and Linda Skitka; I will post a link to an article when one comes out.)

  35. Quantitative vs. Qualitative • The results:

  36. Quantitative vs. Qualitative • That’s 13 (!) people who said they falsify data. What? • The author (Linda Skitka) sent me the data on the next slide when I asked if I could share the results. She thought students might be concerned about this question.

  37. Quantitative vs. Qualitative

  38. Quantitative vs. Qualitative • Note that the “numbers” look one way, but by asking “why?” you get a much more reasonable answer. • Keep this in mind whenever you see numbers. There’s almost always a story, and it’s sometimes very different from the quantitative outcome.

  39. Quantitative vs. Qualitative • She also pointed out the importance of error in interpreting survey research. About 13 out of 545 misunderstood this question. It’s kind of obvious when everyone should say “no,” but what about questions where there’s error, but you can’t tell so easily? • While this link works, here are the results: • https://uic.qualtrics.com/WRReport/?RPID=RP2_bjhdTM0lAkWZMk5&P=CP

  40. I guess this item is so attractive people try to buy it…

More Related