Election Polls In THE 21 st Century: Can they be trusted in the era of cell phones?

1. Election Polls In THE  21st Century: Can they be trusted in the era of cell phones? League of Women Voters Lawrence Township

3. First Statistical Associations • Statistical Society of London (1834) • American Statistical Association (1839) • International Statistical Institute (1885) Sir Ronald Fisher and Prasanta Mahalanobis

4. Anders Kiaer and the Norwegian Census of 1891 Source: Kiaer, Anders N. 1897 “The Representative Method Of Statistical Surveys”, p. 44.

5. Fisher’s Agricultural Research • Use of repeated measurements to estimate crop yields (1919) • Development of randomized experiments, hypothesis testing and tests of statistical significance • Publishes Statistical Methods for Research Workers in 1925

6. Gini, Galvani and the Italian Census of 1921 • Need to make room for the 1930 census • Chose 29 of 216 census districts • Averages computed across districts don’t match population numbers • Both purposive and random sampling considered inferior to full census tabulations Source: Feinberg, Stephen E. and Judith M. Tanur. 1995. “Reconsidering Neyman On Experimentation And Sampling: Controversies And Fundamental Contributions.” Probability and Mathematical Statistics Vol. 15, pp. 47-60.

7. The Impact of the Great Depression • Unemployment clearly a massive problem • Questions on work developed from 1880 through 1930 censuses • Population measurements needed between decennial censuses • Trial Census of Unemployment 1933-34 • Enumerative Check Census of 1937 • Leads to the Current Population Survey Source: Card, David. 2011. “Origins of the Unemployment Rate: The Lasting Legacy of Measurement Without Theory” http://davidcard.berkeley.edu/papers/origins-of-unemployment.pdf

8. 1936 Presidential Election • Literary Digest Poll • 2 million responses • forecasts Republican victory • Gallup Poll • 300,000 responses • forecasts Democratic victory • Roosevelt gets 67% of the popular vote, 98% of electoral votes

9. 1948 Presidential Election • Gallup Poll • Use of quota sampling • 3,250 responses • forecasts Dewey victory • Truman gets 49.5% of the popular vote, 62% of electoral votes • End of quota sampling • Campaign dynamics

10. American Association for Public Opinion Research • Founded in 1947 • More than 2,200 public opinion scientists, survey researchers and statisticians are members • Code of Professional Ethics • Transparency Initiative

11. Average Errors in Presidential Polls 1948-2008 Source: Traugott, Michael. 2005. “The Accuracy of the National Preelection Polls” Public Opinion Quarterly 69(5): 649

12. The 2000 Election: What Went Wrong? • “If we say somebody’s carried a state, you can pretty much take it to the bank.” – Dan Rather • Competitive pressures on networks to call the outcome in Florida first • Voter News Service has problems with following procedure for sampling and interviewing exiting voters and with computer models processing incoming results

13. Accuracy of the 2008 Polls Source: http://electoralmap.net/pollsters/popular.txt

14. Identifying the Likely Voter

15. Likely Voter Questions How much thought have you given to the coming presidential election? How closely have you been following news about the candidates? Do you plan to vote in the presidential election? How certain are you that you will vote? Rate your chance of voting in November on a scale of 10 to 1, with 10 being "definitely will vote" and 1 "definitely will not vote.“ How often do you follow what's going on in government and public affairs? Have you ever voted in your precinct or election district? How often would you say you vote? Do you happen to know where people in your neighborhood go to vote? In the last election, did things come up that kept you from voting or did you vote? Source: Ask the Expert: Determining Who Is a 'Likely Voter' Scott Keeter, Director of Survey Research August 29, 2012

16. Problems with Polling in 2012 • Declining Response Rates • Growth of the “Cell-only” population • Shifting attention to state polls after 2000 • Increasing popularity of survey by “robo-call” Source: http://electoralmap.net/pollsters/popular.txt

17. Declining Response Rates Source: http://www.people-press.org/files/legacy-pdf/Assessing%20the%20Representativeness%20of%20Public%20Opinion%20Surveys.pdf

18. The Growing “Cell-Only” Population Source: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201206.pdf

19. Dual Frame Estimation Summary of Differences Between Dual Frame and Landline Estimates Source: http://pewresearch.org/pubs/1601/assessing-cell-phone-challenge-in-public-opinion-surveys

20. Surveys by Robo-call • Understanding the math behind a survey

21. Rasmussen’s Methodology “While we do not currently call cell phones directly when conducting our surveys, to reach those who have abandoned traditional landline telephones, Rasmussen Reports uses an online survey tool to interview a demographically diverse panel. Just like our telephone surveys, respondents for this approach are selected on a random basis to insure the reliability of the sample. As a result, you cannot sign up to participate. “

22. Aggregating State Polls • Nate Silver’s 538 • RealClear Politics • Princeton Election Consortium • PollTracker • Dave Leip's Atlas of U.S. Presidential Elections

23. RealClear Average

24. Nate Silver/ NY Times

25. Princeton Election Consortium

26. Important Trends • Early voting • Voter ID movement/Voter fraud issue • Demographic changes in the US • Increasing automation/improving robo-call technology • More web-based surveying