The Fruits Of Polling Failure

One last point about the election, and then it’s time to move on:  it’s pretty clear that the entire polling edifice about which modern campaigns, and much of modern political journalism, have been built came crashing down Tuesday night.

poll-public-opinion_001-13The Hill has an interesting article on the degree of polling failure, with a headline stating that pollsters had sustained a “huge embarrassment” as a result of their general failure to predict, or even detect the possibility of, a Donald Trump win.  By way of example, no poll indicated that Trump would win Wisconsin, and instead showed Hillary Clinton with a 6.5 percent lead in that state.  As a result, none of the know-it-all pundits who were pontificating on Election Day even identified Wisconsin as a “battleground state” — when in reality Wisconsin may be the crucial state that handed the presidency to Donald Trump.

I’ve written before about the many judgment calls that go into polling, and how a few tweaks in turnout modeling and the demographic makeup of “likely voters” can change the results.  With this election, we’ve seen the suspicion that polling is not quite as “scientific” as we’ve been led to believe become a painful reality.  Pollsters were just wrong in predicting who would turn out, and in what numbers, and as a result their numbers were skewed — which is why the ultimate results were such a shock.

Polls have become a crutch for campaigns and journalists, and also have been used to crush the aspirations of challengers out seeking to raise money.  Maybe now the “national media” covering the elections will actually get out on the campaign trail, go to events, and report on what the candidates are actually saying and how their audiences are reacting, rather than reporting on polling data and insider leaks about the shape of the horse race.  Maybe now campaigns will pay more attention to what people on the ground are saying and doing, and whether they are responding with enthusiasm to a candidate’s message.  And maybe people deciding which candidate to vote for or financially support will pay attention to the candidates themselves, rather than trying to pick a likely winner based on polling data.

I would never say that this awful election had a positive impact on anything, but if it results in our political processes being much less poll-driven, that would be a step in the right direction.

Poll-Axed

So much of political reporting these days is poll-driven.  A new poll about “likely voters” comes out, and news broadcasts first report on the poll, then report on reaction to the poll, and finally feature a panel of talking heads to blather about “momentum” and “the dynamics of the race” based on the poll results.

But how accurate are those polls, anyway?  Should Hillary Clinton supporters be suicidal because a poll shows Donald Trump ahead in Ohio?  It seems like a new poll or two comes out every day, and the results are all over the map.

screen-shot-2012-10-30-at-11-36-17-pmThe New York Times blog The Upshot decided to conduct a clever experiment to test the role of pollster judgment in analyzing and reporting the results of polling.  The goal was to eliminate the effect of the “margin of error” that we always hear about, and instead focus on the behind-the-curtain decisions pollsters make.  So, The Upshot took the raw data from an actual poll of 867 Florida voters it conducted with pollsters at Siena College, gave that same raw data to four different respected pollsters. and asked them to report the results they drew from the data.

The results of the experiment showed a five percentage point swing in the results reached by the different pollsters, ranging from a four-point advantage for Hillary Clinton to a one-point advantage for Donald Trump, even though the pollsters were reviewing identical data.  Why?  Because the pollsters reached different conclusions about the demographics and characteristics of “likely voters,” and those decisions had dramatic effects on their announced results.  How do you determine who is a “likely voter,” anyway?  Rely on their oath that they’ll be casting their ballot this time?  Make your decision based on their voting history?  Tinker a bit with the breakdown of Democrats, Republicans, and independents, and change the mix of Hispanics, African-Americans, and whites in the “likely voter” population, and you’ve got substantially different results.

My own sense is that this may be the toughest election ever from a polling standpoint.  You’ve got a group of Clinton supporters who are loud and proud in their support for HRC, an apparent mass of ardent Trump advocates lurking below the radar, and then a huge group of disaffected people who really don’t like either candidate and are deciding what to do.  You’ve got lifelong Republicans who are saying, right now, that they won’t vote for Trump, and young people who just aren’t energized by Hillary.  Who among the mass of disillusioned people frustrated by an awful choice is going to vote come November — and for whom?  Based on my interaction with friends and colleagues, most of whom really don’t want to talk about the election, I just don’t see how pollsters can decide that key question with any degree of certainty.

Poll results are interesting, I suppose, but I wouldn’t take them as gospel — particularly in this historically anomalous election.