September 28, 2022
“The polls got it wrong.” We’ve seen this headline far too many times the past few elections. Despite nearly all polls getting it wrong in the 2020 election cycle, our Insights team at LSG was proud to be one of the most accurate pollsters in the last election. We projected Biden would win the popular vote by 5% – coming within less than one percent of the actual outcome of Biden winning by 4.46%.
We conducted this polling as an experiment in 2020 to prove, once and for all, to our corporate, association, and nonprofit clients that polling and market research are still sophisticated and accurate tools to understand the attitudes and behaviors of key audiences. Our modern approach and high-quality sample were essential to ensuring an accurate projection.
As we look ahead to the upcoming November midterm elections and try to make sense of public polling in key states, it is critical to scrutinize polls before taking them at face value. There are several key factors that determine the quality, integrity, and ultimately, accuracy of election polls.
Sampling methodology. Countless election polls are still conducted exclusively by phone. However, the three most accurate polls in the 2020 election cycle, including LSG’s, were all conducted online. Why is online preferable to phone in modern election polling? Voters are less likely to answer their phone and participate in phone surveys than they are online. The notion that Republican or Trump voters will not participate in surveys could not be further from the truth. It is pollsters’ responsibility to reach these voters where they are most comfortable participating.
Moreover, voters are less likely to be honest when they’re asked who they’re going to vote for or how they feel about sensitive policy issues by a live person over the phone. We live in a time where many voters are not necessarily proud of who they are voting for. We survey entirely online whenever possible, because it meets voters where they spend most of their time and better preserves respondent anonymity, ensuring honest responses.
Sample makeup. Most election polls continue to make the mistake of surveying registered voters. Whether we’re talking about elections or public affairs debates, only those who vote matter. Some polls attempt to account for this by surveying self-reported likely voters. This is a step in the right direction but still insufficient to ensure accuracy.
We only survey confirmed likely voters who have voted in at least the last election to ensure high-quality data. All respondents in our panel are matched to their publicly available voter file where we confirm their prior election voting history. While surveying confirmed likely voters is undoubtedly more costly for pollsters, it is essential to ensure a quality sample.
Party identification. Many polls use party registration to classify voters and balance their survey samples. This assumes registered Independents are all swing voters. However, there are serious drawbacks to this limited approach.
Countless voters never bother to update their initial party registration to reflect how they currently vote in general elections. A registered Independent who votes Democratic in every election is not a true Independent; they’re a Democratic voter. We ask voters what types of candidates they typically vote for in order to more accurately represent Democratic, swing, and Republican voters in our sample.
Issue prioritization. Many polls have respondents rate the importance of issues to understand which issues are most important to voters in determining who they will vote for. However, this approach often leads to many issues being considered top drivers of voting behavior without clear differentiation around the relative importance of issues.
For example, a public poll used this approach during the 2020 election and listed COVID-19 as the single most important issue with a statistically insignificant edge over the economy. However, when we asked voters to rank which issue is truly most important, we found that the economy was clearly much more important to voters than COVID-19. Our rank approach helps clients determine what is and is not decisive for the audiences they seek to understand and engage.
This cycle, hate the pollster – not polling – and be sure to dig deeper than polling headlines and toplines when searching for projections in key states.