Phillip Meylan - Nov 12, 2020

Why Were the Polls Wrong?

Since the election on November 3, one question on many people’s minds was how the polling industry could have interpreted the desires of American voters so poorly. While many polls had shown large leads for President-elect Joe Biden in key swing states, the evident reality that played out over the next few days was that pollsters had once more failed to capture the level of support for President Donald Trump. Though this error has not proven as significant as 2016, given Biden’s victories in many key states, the results are much closer than predicted by most major election pollsters and forecasters. How did the polling industry once again misjudge Americans’ voting preferences by such a large margin?

Soul-searching in the industry began almost immediately. Pollsters and forecasters had worked hard to fix their reputation after 2016, but the polling errors this time around are even greater, leaving many looking for explanations. Among potential culprits are a shifting voter base, an overrepresentation of Biden supporters, the idea of “shy” Trump voters, and, perhaps most clearly, a failure to poll the full spectrum of Trump voters. 

This week, The Factual used 17 highly credible articles from 15 outlets across the political spectrum to look at polling errors in the 2020 election to see where the polls went wrong and what this means for the future of the industry. This early post-mortem documents much of the discussion occurring in the media, but it should be noted that a full understanding of the issue will likely take months. 

poll-bann-1

Just How Wrong Were the Polls?

While there were a number of calls that were rather accurate, like Georgia narrowly favoring Biden, these are seemingly the exception rather than the rule when looking at nationwide results. Florida, narrowly forecasted for Biden, ended up going to Trump by about 3 points. Ohio and Texas were thought to be in play for Biden, but went to Trump by roughly 8 and 6 points, respectively. Biden’s polling margins in the pivotal states of Wisconsin and Michigan put him ahead by about 8 points according to FiveThirtyEight, making a victory there seemingly assured before election day; in the AP’s most recent counts, Biden won both by 0.7 and 2.7 points, respectively. In Pennsylvania, arguably the most critical state in the election, the slow counting of mail-in ballots didn’t reveal a Biden victory for days — by about 40,000 votes, or 0.7%, a full 4.1 points off the pre-election polling averages, though these figures may change as the final votes are counted.* 

Note: These numbers are projections based on the remaining votes to be counted.
Source: FiveThirtyEight

The same held true in down-ballot races, where Republicans thwarted Democrats’ efforts to win a Senate majority and even picked up some seats in the House, made possible by results like Susan Collins retaining her Senate seat in Maine against Sarah Gideon, an opponent who led every poll in the weeks running up to the election. In short, polling fell substantially short, locally and nationally. 

Potential Explanations

Some, such as forecaster Nate Silver, have come to the defense of pollsters quickly to argue that such polling errors are fairly common. But for many, the polls are seen to have dropped the ball, sparking debate almost immediately about how the industry could have misrepresented voters preferences by such significant margins. Here are some of the most prominent explanations being discussed in the media and among pollsters.

Changes to the Voter Base: Record-setting turnout this election has served to complicate pollsters’ jobs because there is little precedence for such high turnout numbers, especially in recent history. This means assumptions about how these previously inactive voters will vote equates to a lot of guesswork. For example, high-turnout elections in recent history have tended to be associated with an advantage for Democrats, but the voting surge in this election seems to have benefited both sides, with record numbers of votes for both the president-elect and incumbent. At the same time, there are unforeseen shifts in voter preferences, such as Trump’s unexpectedly high favorability among Latinos. Since polls generally use past voting behavior to help project future voting behavior, they can be susceptible to missing significant or rapid shifts in voter preferences.

“In other words, Mr. Trump won several key states in the election in part because he changed the electorate, a possibility the pollsters had signaled but which the poll’s results didn’t fully reflect.” - Aaron Zitner, Wall Street Journal

Overrepresentation of Biden Voters: Another potential explanation comes from overrepresentation of Biden voters. As Nate Cohn posits for the New York Times, high levels of political engagement related to the Black Lives Matter movement or women's rights likely drove more progressive voters to be politically active, resulting in more poll responses among that group. Similarly, the Covid-19 pandemic seems to have driven an uncharacteristically high response rate from Democratic voters, who were both likely to be abiding by stay-at-home guidance and to be seeking to voice political frustrations during an arguably mismanaged pandemic. And contrary to projections that Covid-19 would hurt Trump, Cohn notes that Trump “fared slightly better in places with high coronavirus cases than in places with lower coronavirus cases.”

“The theory is that the kind of people who answer polls are systematically different from the kind of people who refuse to answer polls — and that this has recently begun biasing the polls in a systematic way.” - Dylan Matthews, Vox

Source: New York Times

“Shy” Trump Voter Theory: This theory, a much-discussed explanation for polling errors in 2016, is basically that Trump voters will actively choose to tell pollsters that they favor Trump’s opponent in an effort to avoid being viewed as socially undesirable. Many pollsters would agree that their polls have failed to register support for Trump, but most cast doubt on the shy voter theory. Indeed, a report from the American Association of Public Opinion Research found no evidence of such voters in 2016, nor do polls in 2020. However, this hasn’t stopped proponents of this theory from putting it forward. The Trafalgar Group, a pollster which used this theory to project 2016 rather effectively, also made solid predictions this time around, including Wisconsin (+1 for Biden) and Florida (+2 for Trump), but they notably also projected Georgia going to Trump by 5 points, where Biden leads by just over 10,000 votes. This variability suggests that the theory about shy voters is probably not the explanation for inaccurate polling, or at least that there are other overriding factors at play beyond specific voters hiding their preferences, but the theory will surely receive further scrutiny.

Failure to Poll Trump Voters: Perhaps the most likely explanation for the polling errors is that the polls simply failed to create truly representative samples. The thinking here goes that Trump supporters are highly skeptical of authority — be it the media, science, or academia — and therefore are less likely to participate in polls generally. As part of this, moderate Trump supporters may be both more likely to participate in polls and more likely to defect politically given their more moderate viewpoints. As a result, pollsters found that they significantly overpredicted the number of former Trump voters that would support Biden and underpredicted the number of voters intending to vote for Trump. This is something that pollsters anticipate and try to correct for, for example, by adjusting for the level of education among respondents or by engaging with other forms of polling beyond traditional landline surveys (especially given the declining use of landline phones), such as in person, online, or through text messaging, but it seems that such measures this year were entirely insufficient.

“If it turns out that support for right-wing populism is independently correlated with survey response rates, then it could be impossible for pollsters to get an accurate count no matter how they weight the demographics.” - Gilad Edelman, Wired

The Future of Polling

Polling got a lot of this election wrong, and led to misdirected journalism as a consequence, but that doesn’t mean it’s time to throw in the towel on the whole industry. As pollsters adamantly argue, they are not meant to be predictors, merely a snapshot of attitudes that are subject to the complexity of the entire voting populace and shifting preferences. While they were off the mark this election cycle, they are essential to understanding the thoughts and desires of the people and did, to some degree, give us a sense of which direction the election was likely to trend. 

“The inability to rely on polling also undermines a range of generic political choices. Many voters decide which candidate they favor in primary elections based in part on rational calculations about who is most likely to win a general election. Policy makers choose what ideas they support or oppose based on what they think the public supports. If poll data are not reliable, though, the candidates that triumph in primaries are more likely to be the loudest and best-funded, and the policies politicians back might run contrary to the wishes of the voters who elected them.” - David A. Graham, The Atlantic

Polls are also critical outside of elections, where favorability for and against specific policy choices indicates to politicians where the realm of possibility lies for lawmaking. For example, voters this election indicated their strongest priorities are the economy, racial inequality, and the Covid-19 pandemic. Absent of these signals, policymakers would be left flying blind. Flying with a rough estimate of where to go is better than having no sense of direction at all. It’s up to pollsters and forecasters to improve their methods and make that picture clearer once more.


* These margins of victory are preliminary until recounts are completed and any allegations of voter fraud, and the impact they had, are resolved. However, the final tallies are not expected to change materially, so this analysis uses the existing AP numbers.

Appendix

This appendix shows each of the articles used to inform the findings of this article, as well as how the articles scored according to The Factual's credibility algorithm. To learn more, read our How It Works page.

Title

Author

Publisher

Publisher Political Bias

Credibility Grade

Left

The Polls Underestimated Trump — Again. Nobody Agrees on Why.

Giovanni Russonello

New York Times

Moderate Left

82%

Pollsters face another reckoning this year, but the reasons could differ from 2016

Laura Feiner

CNBC

Moderate Left

80%

How wrong were Florida polls on Trump and Biden?

Langston Taylor

Tampa Bay Times

Moderate Left

73%

One pollster’s explanation for why the polls got it wrong

Dylan Matthews

Vox

Left

73%

So How Wrong Were the Polls This Year, Really?

Gilad Edelman

Wired

Moderate Left

72%

The polling industry blows it again

Steven Shepard

Politico

Moderate Left

71%

What Went Wrong With Polling? Some Early Theories

Nate Cohn

New York Times

Moderate Left

70%

The Polling Crisis Is a Catastrophe for American Democracy

David A. Graham

The Atlantic

Moderate Left

70%

The Polls Weren’t Great. But That’s Pretty Normal.

Nate Silver

FiveThirtyEight

Moderate Left

65%

Why polling failure is often journalistic failure

W. Joseph Campbell

The Hill

Moderate Left

65%

Center

Another Election, Another Round of Poll Bashing. Is That Fair?

Michael Schulson

Undark

Center

90%

Key things to know about election polling in the United States

Courtney Kennedy

Pew Research

Center

84%

What went wrong with the 2020 election polls and what’s next for political polling?

Kelly McBride

Poynter

Center

74%

Right

The Economy, Racial Inequality, and COVID-19 Topped Voter Minds

J.D. Tuccille

Reason

Moderate Right

78%

Shy Trump voters' share why they never made their support public: 'I don't want to get beat up'

Michael Lee

Washington Examiner

Moderate Right

73%

The Media Had 4 Years to Figure Out Trump Voters. They Blew It.

Robby Soave

Reason

Moderate Right

68%

What Went Wrong With the Polls This Year?

Aaron Zitner

Wall Street Journal

Moderate Right

N/A

 

Written by Phillip Meylan

Phillip is a writer, editor, and researcher. Before completing his MSc in Comparative Politics at the London School of Economics and Political Science in 2019, he worked as an editor and content strategist for the Center for Strategic and International Studies in Washington, D.C. When he’s not working, you can find him playing soccer, hiking, or cooking.

    Get the most credible stories on the hottest news topics

    The Factual uses its Credibility Grade to find and deliver the most credible stories across the political spectrum on trending topics.

    1*AESeEuEnns8-Lo8_Bt9_JA