1 / 24

Polling 101

Polling 101. A Practical Guide for Campaigns. Gaurav Oberoi CEO, PrecisionPolling.com gaurav@precisionpolling.com April 25, 2010. What We’ll Talk About. Overview Methodology Campaigns and Polling. Overview. What is public opinion? Who cares? How can we measure it?.

ita
Download Presentation

Polling 101

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Polling 101 A Practical Guide for Campaigns Gaurav Oberoi CEO, PrecisionPolling.com gaurav@precisionpolling.com April 25, 2010

  2. What We’ll Talk About • Overview • Methodology • Campaigns and Polling

  3. Overview What is public opinion? Who cares? How can we measure it?

  4. What is Public Opinion? “The aggregate of individual attitudes or beliefs held by the adult population.” • Wikipedia “… those opinions held by private persons which government finds it prudent to heed.” - V.O. Key Jr.

  5. Who Cares and Why? • Government • How well am I doing? • What else do I need to focus on? • Candidates • What do voters want? • How do I market myself? • Am I going to get elected? • Corporations • How can I leverage public opinion for my own goals? • Media • How can I get more pageviews? (or sell dead trees?)

  6. How Can We Measure It? • Ask everyone what they think • Haha, that’s good. • Ask as many people as I know • But that’s clearly not representative • Ask a random sample of people that represent the population at large, and extrapolate • Aha! That sounds like science!

  7. Methodology What does scientific polling look like? How do we know it’s accurate? Where does it break down?

  8. 5 Steps to a Scientific Poll • Define the population to be sampled • E.g. voters for the upcoming election • Construct a sample • E.g. 20k likely voters with match demographics • Write some questions • Careful to minimize bias in your wording • Pick a survey methodology and run it • Live interviewers, automated, online… • Analyze and compute results • E.g. calculate and report error, weight if necessary…

  9. 1: Define Population to Sample • Usually: those who will vote in the election • But not always so easy: • What if unexpected voters show up?E.g. lots of young voters and Obama • The answer: a combination of trending + gut • Who has typically turned up to vote in the past? • Anything that might materially change that?

  10. 2: Construct a Sample • Randomly pick people who match the demographics of the population • E.g. if your population of likely voters are 55% female and 45% male, your sample should be too • There will be some sampling error • This is the +/- margin of error you see on polls • E.g. sample (error): 100 (11%), 600 (5%), 2k (2%) • Picking a good sample has other complexities • E.g. account for non-response (don’t answer phone)

  11. 3: Write Questions • Goal is to gather data while minimizing bias • Basics: • Include all options, no leading questions etc. • More nuanced: • Randomly rotate names; try to match what will eventually be on the ballot; identifying the sponsor may bias answers etc.

  12. 4: Pick a Survey Methodology • Most common: live interviewers on a phone • CATI (computer assisted telephone interviewing) • Pros: can ask complex surveys, verify speaker on the phone, works for long surveys • Cons: expensive, quality varies across interviewers, social desirability bias, costs many $$$, slow • Increasingly common: automated phone polls • Pros: cheap, fast, removes social desirability bias (Bradley effect), uniform quality • Cons: can’t work for long or complex surveys, can’t be 100% sure who’s on the other end • Not common: online or panel polls

  13. 5: Analyze and Compute Results Typically involves: • Computing sampling error • Generating crosstabs by demographic • Weighting to account for non-response biasE.g. not enough young people answered my poll • Scratching heads if there are a lot of undecideds • Writing analysis with regards to political context

  14. BUT What About These Problems? • Non-response bias: • Hang up  low response rate so sample is skewed • Typically: few young people answer • Social desirability (Bradley effect): • “Of course I’m going to vote, I’m a good citizen!” • Cell phone problem • “We can’t poll cell phones?! But that’s all I have!” • Lots of undecideds • Build models to guess what they will do?

  15. Campaigns and Polling How do campaigns use polling? How much does it cost? What are some practical tips?

  16. How do Campaigns Use Polling? • Understanding voter sentiment • “What do they care about most?” • Testing messages • “What’s the best way to market my guy/gal?” • Measuring media spend • “Did that direct mailing word? If so, let’s do more.” • Responding to hot button issues • “The markets just crashed, how should we respond?”

  17. How do Campaigns Use Polling? (Cont.) • Horse race polls • “Did we get more of the swing vote this week?” • Micro-targeting • “With my limited budget, where can I make the most impact?” • Influence • “This poll is super positive, send it to Publicola!”

  18. And How Campaigns Use FAKE Polling • Fake polls: called “polls” but are mostly ads • Typically frowned upon as shady • Push polls: • One dimensional questions to influence you • E.g. “Do you want govt. bureaucrats deciding when you can see your doctor?” • Don’t confuse this with message testing • Straw polls: • Non-scientific/informal poll to a select audience • E.g. if The Stranger was to ask its readers about legalization and report on it to gain influence

  19. How Much Does it Cost? • Live interviewer polls are expensive • $10k-$35k • 20 minutes; 20-30 questions; 500-1000 completes • 1 week to field; 2-3 weeks for results and analysis • Automated polls are much cheaper • $1-4k • 3-7 minutes; 5-15 questions; 500-1000 completes • Days to setup and get results • Hours if you use PrecisionPolling.com

  20. Polling in the Campaign Lifecycle • In the beginning • Commission a big live interviewer poll ($10-35k) • Get a lay of the land; strategy and messaging • Before and after direct mail or TV • Polls to learn who to target; measure spend after • Automated polls work well ($1k) • Last few weeks • Nightly tracking polls; automated ($1k)

  21. Practical Tips for Polling • Poll even if your budget is tiny • Do an automated poll if that’s all you can afford • Data can focus your ad spend = more value • Data can help move the out of a deadlock E.g. Bob & Lisa disagree about the mailer message • Data can cover your you-know-whatE.g. candidate disagrees, but data backs you up

  22. Practical Tips for Polling (cont.) • Keep costs low by improving response rates • Make sure your caller-id is local • Call in the early-evening (6:30-8:00pm) • Have a snappy intro • Use clear and loud voices, or recognizable ones • Simple sentences; foreign language if it helps

  23. Practical Tips for Polling (cont.) • Use high quality lists • Meaning: updated for moving and deaths, scrubbed for do-not-call, good matches to phone numbers etc. • Good phone list vendors keep them up-to-date • Ask around for recommendations • Leverage your friendships • Your party may maintain phone lists you can use • Your consultant may be able to help with basic polling

  24. Practical Tips for Polling (cont.) • Understand legal issues: • Disclosure: do I need to say who’s paying for this? • Do not call list: can I make calls in this state? • Automated: how about automated ones? • And more: legal requirements vary based on audience, message, and funder.

More Related