Polling the Leadership Debates The role of the instant polls Andrew Hawkins & Caroline Lawes

1. Polling the Leadership DebatesThe role of the instant pollsAndrew Hawkins & Caroline Lawes 22nd November 2010

2. Overview • The players • Methodology & sampling • Challenges • Results • Impact • Lessons learnt

3. The players • Angus Reid • ComRes/ ITV News • ICM/ Guardian • Populus/ The Times • YouGov/ The Sun

4. The methodologies • Across the different companies, three methodologies were used and three types of weightings. • Automated telephone • Online • Telephone • Nationally representative weighting • Viewer profile weighting • No weighting

5. Telephone • ComRes: Automated Telephone • Pre-recruited panel of 10,000 people, demographics collected in advance • Programmed key touch survey using James Mates’s voice • Quotaed sample to send out • Automatic sample weighting – nationally representative • Sample size of 4032, 2691, 2372 • Six minutes fieldwork time • ICM: CATI methodology • Random sample of omnibus for recruitment. • Recontacted sample of people who said they would be watching. • Sample of 500 on the evening after the debate. • Weighted to the profile of all people selected at random who previously stated they would be watching the debate.

6. Online • Populus: Online • Recruited through a nat rep online panel. • Reminders the day before and again on the morning of the debate.  • Data were not weighted, but was sub sample of nat rep base. • Sample of 1000 • YouGov: Online • Sample of people who said they would definitely or probably watch the debate, from large screener • Weighted to be representative of debate viewers. • Reminder emails before the debates • Sample of 1000 • Angus Reid: Online • Sample: panelists from Springboard UK panel who watched debate • Fieldwork on evening and next day • Weighted to be nationally representative • Sample of 1000-1500

7. Audience profile As a UK first, a key question was who will watch the debates? Actual viewing figures:

8. Weighting In hindsight we know what the audience looked like, but each instant poll had to make a choice of whether to and how to weight.

9. Impact of weights Following the release of the viewing figures we reweighted the data to the actual viewer profile and the differences are given below. Who do you think won tonight’s debate?

10. The Worm • 20 participants across the two marginal constituencies of Bolton West and Bolton North East • Respondents were definitely going to vote, but were undecided who for. • There was a good spread of men and women, different ages and ethnic groups and different past voting intentions. • Participants use an electronic keypad where they could let us know when they liked something, when they didn’t like something, or when they were neutral to what was being said.

11. Challenges • Sampling: What is the best way to recruit, how can we be sure people are planning to watch debates, how should invitations be quotaed? • Target weights: The viewer profile before the debates was unknown so how is the decision made to weight the results, or is the sample self selecting? • Speed: Allowing for the over-run of the programme, yet not wanting viewers to be influenced by commentary

12. Results

13. Additional questions • After first debate, Conservatives, closely followed by Lib Dems, had best policies on immigration and this topic became increasingly important during the campaign. • In debate two, expectations had been set. Indeed 47% of viewers thought that Brown performed better than expected in this debate, compared to just 21% of people for Clegg. • 43% of viewers thought Clegg gave the most honest answers – compared to 29% for Cameron and 23% for Brown. • After debate three, viewers were still split on who would be best to make spending cuts with about a third of people choosing each party. • Source: ComRes

14. Changing opinion Most evasive and least willing to give straight answers Source: YouGov Most like to see as PM Source: Populus

15. The impact • Debates became the defining feature of the campaign • Liberal Democrat surge • Financial markets quivering at prospect of hung parliament

16. Impact on the polls Average daily voting intention figures from all polls released

17. Financial markets • The prospect of a hung parliament caused the value of the pound to drop. • 16th April intra-day figures show the impact of the instant polls from debate one. GB Pound v. US Dollar on 16th April 2010

18. Lessons Learnt • Sampling: Measuring what viewer profile may look like before the event. • Weighting: What is the most accurate weighting and how can the national impact be modelled? • Timing: Impact of polling too early or too late • Interpretation: Ensuring the results are not misinterpreted when published in the heat of an election campaign • The leadership debates are likely to be a feature of all future General Elections and the polls have a vital role to play

19. For further information please contact: Caroline Lawes ComRes, Four Millbank, London SW1P 3JA 020 7340 9637 caroline@comres.co.uk