Exit Polls – the Whys and Wherefores (and more importantly the Hows)

Exit Polls – the Whys and Wherefores (and more importantly the Hows) Nick Moon, GfK NOP

My own experience • Extensive research to develop an exit polling technique for the BBC • Ten Parliamentary by-elections • 1990 European elections • Four GB general elections – 1992 – 2005 • One Greek general election • On-site observation of US Presidential Election 2000 • The most recent Ukrainian Presidential Election

Opinion Polls and Exit Polls • Methods for conducting opinion polls can differ enormously from one country to another • Face to face/telephone/online • Random/quota • Methods for conducting exit polls are much the same the world over • Ukraine is a vast country (with even worse weather) but there exit poll was almost exactly the same as ours

Cut to the Chase, Nick • Don’t just make excuses about Ukraine being a big country – tell us how you got on

While the Triumph is Still Fresh Result Poll Error Yanukovych 35.3 34.5 -0.8Tymoshenko 25.1 25.6 +0.5 Tigipko 13.1 13.8 +0.7 Yatsenyuk 7.0 7.0 0Yuschenko 5.5 5.6 +0.1

What is an Exit Poll? • A very expensive way of getting an idea of an election result a few hours before you know for sure • A major contribution to understanding of the electoral process • A way to tell if the election was conducted fairly • (or is it?)

Types of Exit Poll • Prediction Poll • Who is going to win and by how much? • Analysis Poll • Why did voters vote the way they did?

An exit poll may be more than a poll • Interviewing voters • This is all we do in the UK • Collection of precinct-level results as soon as they are announced • A representative sample can predict the whole, quickly • In some European countries this is all that is done • In the US this is done alongside a voter survey • Analysis of partially aggregated actual results • Used to fine-tune projections made using the above

Exit Poll Problems - issues • Interviewing has to be clustered by polling station • Cost is a function of number of stations, not number of interviews • Voting behaviour tends to be very clustered at the polling station level • Several cases in the past of >90% support for one party • And several of 100% in Ukraine • Far more interviews are conducted than are actually needed, just because the interviewer might as well do as many interviews as are easily achievable • Actual results at polling station level not always available, which makes sampling less reliable • US and Greece yes, UK and Ukraine no

Exit Poll Problems – issues (UK) • We are really predicting the result of over 600 separate elections, and can’t cover each one • The only local level information available on actual voting is at ward level, and only for local elections • Local level turnout is far lower and behaviour often very different • This makes stratification of the sample by voting is virtually impossible

Exit Poll Problems – UK solutions • Use as many polling stations as possible • Use the same polling stations as the previous election • Can look at change rather than simple totals • Eliminates effect of bias IF bias is constant (Big IF)

The 2005 GB Exit Poll • Conducted jointly by NOP and MORI • Conducted on behalf of both BBC and ITN • 120 polling stations • Most used previously by NOP or MORI • Approx 16,500 interviews in all • Just measuring claimed current vote

The Practicalities (1) • Polling runs from 7am to 10pm • Split into two equal shifts • Each has an interviewer and a “clicker” • “Clicker” uses preordained sampling interval to select respondent • Interviewer estimates voting behaviour, then asks voter to fill in an exit poll ballot • No replacement for refusals

The Practicalities (2) • Ballot designed to replicate actual one as closely as possible • Has candidate names and parties • In alpha order as on real ballot • Ballots collected every hour • Individual results phoned back to team of telephone interviewers • Results weighted to take into account any discrepancy between intended and actual number of interviews in each hour

The Practicalities (3) • Aggregate data at level of time by polling station produced at regular intervals • Along with individual level data file • Collected by the BBC/ITN analysts • Run through various algorithms • Initial projection of result at 10pm • Final projection at 11pm

And How Good Was It?

In case you’ve forgotten

The BBC/ITN exit poll • Forecast majority 66 seats • Actual majority 66 seats

The BBC/ITN exit poll Jeremy Paxman (paraphrased) “I don’t know why we spend all this money on the exit poll, we should just do without it” Tony King (paraphrased) “These results suggest our exit poll is overestimating Labour”

The BBC/ITN exit poll Predicted majority based on exit poll 66 seats Predicted majority based on early actual results 80 seats Final majority 66 seats

The BBC/ITN exit poll Sorry Jeremy The BBC should carry on paying for the exit poll, and stop paying for everything else

The 2010 General Election Exit Poll • Conducted jointly by GfK NOP and Ipsos MORI • BBC and ITN now joined by Sky • As close as possible to the 2005 design • Changed polling places where the constituent voters or the place itself had changed significantly • Replaced these with an eye to filling gaps in current design and also looking to the next election

Are exit polls measures of election fixing? • Ukraine 2004 Round 2 • But many other indicators of fraud • Venezuela • Questions raised about exit poll • US 2000 • There almost certainly was some fraud but the exit poll is not a smoking gun

To conclude • Exit polls are a good way to give a good advance idea of who has won, and why, and properly conducted exit polls can act as some sort of check on electoral fraud, but as President Carter said, they should not be used as sole arbiters of the fairness of an election

Exit Polls – the Whys and Wherefores (and more importantly the Hows)