So much of political reporting these days is poll-driven. A new poll about “likely voters” comes out, and news broadcasts first report on the poll, then report on reaction to the poll, and finally feature a panel of talking heads to blather about “momentum” and “the dynamics of the race” based on the poll results.
But how accurate are those polls, anyway? Should Hillary Clinton supporters be suicidal because a poll shows Donald Trump ahead in Ohio? It seems like a new poll or two comes out every day, and the results are all over the map.
The New York Times blog The Upshot decided to conduct a clever experiment to test the role of pollster judgment in analyzing and reporting the results of polling. The goal was to eliminate the effect of the “margin of error” that we always hear about, and instead focus on the behind-the-curtain decisions pollsters make. So, The Upshot took the raw data from an actual poll of 867 Florida voters it conducted with pollsters at Siena College, gave that same raw data to four different respected pollsters. and asked them to report the results they drew from the data.
The results of the experiment showed a five percentage point swing in the results reached by the different pollsters, ranging from a four-point advantage for Hillary Clinton to a one-point advantage for Donald Trump, even though the pollsters were reviewing identical data. Why? Because the pollsters reached different conclusions about the demographics and characteristics of “likely voters,” and those decisions had dramatic effects on their announced results. How do you determine who is a “likely voter,” anyway? Rely on their oath that they’ll be casting their ballot this time? Make your decision based on their voting history? Tinker a bit with the breakdown of Democrats, Republicans, and independents, and change the mix of Hispanics, African-Americans, and whites in the “likely voter” population, and you’ve got substantially different results.
My own sense is that this may be the toughest election ever from a polling standpoint. You’ve got a group of Clinton supporters who are loud and proud in their support for HRC, an apparent mass of ardent Trump advocates lurking below the radar, and then a huge group of disaffected people who really don’t like either candidate and are deciding what to do. You’ve got lifelong Republicans who are saying, right now, that they won’t vote for Trump, and young people who just aren’t energized by Hillary. Who among the mass of disillusioned people frustrated by an awful choice is going to vote come November — and for whom? Based on my interaction with friends and colleagues, most of whom really don’t want to talk about the election, I just don’t see how pollsters can decide that key question with any degree of certainty.
Poll results are interesting, I suppose, but I wouldn’t take them as gospel — particularly in this historically anomalous election.