There\u2019s a widespread feeling that polling failed to predict the 2016 election results. Do you agree?<\/strong><\/p>\n\n President Trump\u2019s victory certainly caught many people by surprise, and I faced more than one Hillary Clinton supporter who felt personally betrayed by polling. But the extent to which the expectation of a Clinton victory was based on flawed polling data \u2013 or incorrect interpretation <\/em>of polling data \u2013 is a big part of this question.<\/p>\n\n Polling\u2019s professional organization, the American Association for Public Opinion Research<\/a> (AAPOR), spent the last several months looking at the raw data behind the pre-election<\/a>\u00a0polls in an attempt to answer this question. (Note: The leader of the AAPOR committee tasked with this inquiry is Pew Research Center\u2019s director of survey research, Courtney Kennedy<\/a>.)<\/em> While it might surprise some people, the expert analysis found that national polling in 2016 was very accurate by historical standards. When the national polls were aggregated, or pulled together, they showed Clinton winning among likely voters by an average of 3.2 percentage points. She ended up winning the popular vote by 2.1 points \u2013 a relatively close call when looking at presidential polling historically, and significantly closer than what polls suggested in 2012.<\/p>\n\n Of course, as we all well know, the president isn\u2019t chosen by the national popular vote, but by the Electoral College \u2013 so it is the state poll results, rather than the nationwide surveys, that were particularly relevant for those trying to project election results. And the state polls, according to the report, \u201chad a historically bad year.\u201d In particular, in several key Midwestern states with a history of voting Democratic at the presidential level \u2013 including Wisconsin, Michigan and Pennsylvania \u2013 Clinton was narrowly ahead in pre-election polls, only to be edged out on Election Day. So what happened?<\/p>\n\n The AAPOR committee report offers at least two factors that were at play. First, data suggests<\/a> that a number of voters made up their minds in the final days of the campaign. And those late deciders broke for Trump by a significant margin. In the battleground of Wisconsin, for example, 14% of voters told exit pollsters that they had made up their minds only in the final week; they ultimately favored Trump by nearly two-to-one. Yet nearly all of the polling that drove expectations of a Clinton victory in Wisconsin was conducted before the final week of the campaign, missing this late swing of support.<\/p>\n\n Secondly, unlike in other recent elections, there was a stark education divide in candidate support that some state-level polls missed. Pollsters have long talked about the importance of gender, religion, race and ethnicity as strong correlates of voter preference. Last year, education was also a strong correlate. A number of pre-election polls that didn\u2019t account for it \u2013 by adjusting, or \u201cweighting,\u201d their samples to better reflect the full population \u2013 were off if they had too many highly educated voters, who tended to vote for Clinton, and too few low-education voters, who tended to vote for Trump.<\/p>\n\n What does that all mean? Well, polls may have been an accurate representation of voter preferences at the time they were taken. But preferences can change, particularly in a fast-moving campaign. Combine that with an education gap that wasn\u2019t apparent in other recent elections \u2013 and wasn\u2019t reflected in some state-level surveys \u2013 and you can see why some of those state polls did a poor\u00a0job of projecting the ultimate outcome. The key for survey practitioners is that both these types of errors can be addressed by known methods.<\/p>\n\n If polls can get election outcomes wrong, doesn\u2019t that mean polling in general is unreliable?<\/strong><\/p>\n\n No. There are important differences between election polling and other kinds of survey work.<\/p>\n\n Forecasting elections doesn\u2019t just involve asking people whether they support candidate A or candidate B. It also involves trying to determine whether respondents will act on their preferences by casting a ballot at all. And that extra step of identifying \u201clikely voters\u201d has long been one of the most challenging things for pollsters to do. Survey respondents generally do a better job of telling you what they think than what they are going to do, especially when it comes to voting. It is this extra step \u2013 where a lot of assumptions about factors associated with turnout need to be made \u2013 that is quite distinct from the principles of random sampling and good question design that make survey research valid and reliable.<\/p>\n\n Another wrinkle when it comes to election polling is that we\u2019re now in an era when the aggregation of polls emphasizes their use specifically as a forecasting tool, and asserts degrees of certainty to those forecasts. This is much like the weatherman using a variety of tools to forecast the weather.\u00a0But I don’t think this is the primary, or even a good, use of polling.<\/p>\n\n In fact, most survey work is not<\/em> engaged in election forecasting. Instead, it\u2019s meant to get beyond the surface and into people\u2019s heads \u2013 to truly understand and explain their values, beliefs, priorities and concerns on the major issues of the day. These kinds of surveys are aimed at representing all <\/em>citizens, including those who might not vote, write to their member of Congress or otherwise participate in the political process.<\/p>\n\n After each election, there is a tendency for the winning candidate to claim a mandate and point to the results as evidence of the public\u2019s will. But given that so many citizens don\u2019t vote, and that many who do vote don\u2019t like either of the options before them, elections aren\u2019t necessarily reflective of the will of all people. A deep, thoughtful survey can help address this disconnect by presenting the voice of the public on any number of issues.<\/p>\n\n Let\u2019s talk about response rates. In recent decades, fewer people have been responding to telephone surveys. Does that mean poll results are becoming less accurate?<\/strong><\/p>\n\n We spend a lot of time worrying about declining response rates. There is no doubt that the share of Americans who respond to randomized telephone surveys is low and has fallen over time \u2013 from 36% of those called in 1997 to 9% in 2016. A low response rate does signal that poll consumers should be aware of the potential for \u201cnonresponse bias\u201d \u2013 that is, the possibility that those who didn\u2019t respond may be significantly different from those who did participate in the survey.<\/p>\n\n But a Pew Research Center report released last month<\/a> shows that survey participation rates have stabilized over the past several years and that the negative consequences of low response rates for telephone survey accuracy and reliability are limited. In particular, there\u2019s no evidence that Democrats or Republicans are systematically more or less likely to participate in telephone surveys.<\/p>\n\n There\u2019s also a misperception that the rise of mobile phones is a problem for survey accuracy, but it\u2019s not. To be sure, mobile phones are a factor: It\u2019s now estimated that roughly 95% of U.S. adults own a mobile phone, and more than half<\/a> live in households that have no landline phone at all.<\/p>\n\n![\"\"](\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/2017\/06\/Dimock_Mike-4_160x240.jpg\")
<\/a><\/figure>\n\n