Exit-poll analysis and prediction

1. Exit-poll analysis and prediction Stephen Fisher Following: Curtice, John and David Firth (2008) Exit polling in a cold climate: the BBC-ITV experience in Britain 2005, J. R. Stat. Soc. A, 171, 509-539. Presentation to the NCRM-BPC Opinion Polls Conference, British Academy, 20th January 2010

2. Broad research design principles: 1 • Model the pattern of change across constituencies in the share of the vote since the last election • i.e. not directly estimating the results in 2010 but change since 2005, since more variance between constituencies in shares than in changes. • Also, assessing and allowing for different swings in different seats

3. Broad research design principles: 2 • Estimate probabilities for each party winning each seat • i.e. allow for the random/unexplained variation between constituencies in the prediction • Predicted number of seats for each party is the sum of the probabilities across constituencies • Primary aim is to predict seat totals, not share of the vote

4. Infrastructure • 2010 exit poll will be a joint BBC/ITN/Sky project • Fieldwork by MORI and NOP, as in 2005 • People contributing to the analysis: Jouni Kuha (LSE), John Curtice (Strathclyde), Clive Payne (Oxford), Rob Ford (Manchester) • Debt of gratitude and computer code to David Firth

5. Selection of constituencies • Revisit all 107 viable locations from 2005 • Top up to 130 by sampling the kinds of constituencies thought to be useful and currently under-represented. • e.g. new decision to explicitly attempt to have a group of Lab-LD seats • Pick the most representative polling station in the constituency for the new locations

6. Exit poll locations in 2005

7. Statistical analysis • Model the change in the share of the vote since the last election • Consider data with and without interviewer guesses for those who refused • In 2005 ignoring the guesses and refusals worked best • Consider lots of different predictor variables • E.g. census data, market research data, strategic situation, incumbency • Expectations of geographical variation informed by practice with pre-election day polls • But keep the final model simple (N=130)

8. Producing predictions • Generate predicted probabilities for each party winning each seat from the statistical regression models of the data. • Using estimates of both explained and unexplained variance. • Sum the probabilities for each party across constituencies to estimate the total number of seats for the party. • In 2005 the exit-poll data suggested a Lab majority of 100 under uniform change, but the method accurately predicted 66 • Introduction of explained (regression) and unexplained (probabilistic prediction) variation both equally account for the difference between uniform change and final prediction.

9. Probabilistic prediction compared with the swingometer • What would be the effect of allowing for unexplained variation in a swingometer estimate of the result? • Depends on the distribution of seats according to marginality. • Smoothes the relationship between predicted swing and predicted seats.

10. Need for probabilistic prediction in 2005

11. Marginal Lab-Con seats for 2010 E.g. probabilistic method would predict fewer Con seats from a 7% swing than the swingometer because it would allow for the seats immediately either side of the 7% point to split between Con and Lab, and there are more to the left than right. -But not much difference.

12. Simulation 1: Stability of notional 2005 results • Rerun 2005 notional results but adding noise to allow probabilistic results • Seats with 05 margin <1% become 50:50 • Seats with 05 margin c.4% become 90:10 • Changes the seat totals from Con 210, Lab 348 to Con 212, Lab 346 with LD unchanged. • i.e. not much change

13. Simulation 2: Poll projection • ukpollingreport.co.uk average of polls: • Con 41, Lab 29, LD 18 • i.e. +8, -7, -5 since 2005 • Uniform swing: Con majority of 44 • Probabilistic projection: Con majority of 48 • Very little difference

14. Simulation 2: Seats in the balance for the Tories under the simulation constituency wp05 sp05 pctmaj05 conprw labprw ldprw Dagenham & Rainham LAB CON 15.7 0.41 0.59 0.00 Erewash LAB CON 15.7 0.42 0.58 0.00 Norwich South LAB LD 7.4 0.43 0.57 0.00 Bath LD CON 13.6 0.43 0.00 0.57 Leeds North East LAB CON 15.5 0.44 0.56 0.00 Crewe & Nantwich LAB CON 15.5 0.44 0.56 0.00 Ochil & South Perthshire LAB SNP 1.5 0.45 0.00 0.00 Oxford West & Abingdon LD CON 13.4 0.45 0.00 0.55 Newport West LAB CON 15.3 0.47 0.53 0.00 Warwickshire North LAB CON 15.3 0.47 0.53 0.00 Hampstead & Kilburn LAB LD 1.1 0.47 0.21 0.32 Coventry South LAB CON 15.2 0.48 0.52 0.00 Dorset Mid & Poole North LD CON 13.1 0.49 0.00 0.51 Argyll & Bute LD CON 13.0 0.50 0.00 0.50 Telford LAB CON 15.0 0.50 0.50 0.00 Berwickshire, Roxburgh & Selkirk LD CON 13.0 0.50 0.00 0.50 Winchester LD CON 12.7 0.53 0.00 0.47 Luton South LAB CON 14.7 0.54 0.46 0.00 Brighton Pavilion LAB CON 13.1 0.57 0.26 0.00 St. Austell & Newquay LD CON 12.4 0.57 0.00 0.43

15. Simulation 2 – Distribution of Predicted Probabilities

16. Particular difficulties for exit poll prediction in 2010 • Boundary changes • Possible errors in both dependent and explanatory variables • Expenses scandal • More MPs stepping down • Potentially more variance between constituencies • Census data old (2001) • High expectations from 2005!