The Analysts Failed Badly In 2016, They Look Like They’re Going To Fail Again In 2020

In a 1991 James Fallows wrote a piece in the Atlantic trying to explain what exactly he found so obnoxious about the magazine, the Economist. To do this he related an anecdote about a government official going on to him about about he’d just read on Japan’s economic miracle. This book, he told Fallows, extensively proved that Japan had succeeded by bucking interventionist economic policies and embracing free markets. As it happened, Fallows was very familiar with the book, and knew it was making the exact opposite argument.

This official, it was implied, had drawn his opinion from a review written in the Economist, which made shallow, superficial reading of the book which badly mangled its meaning until black was white. The Economist was trying to hammer the book into conformance with its own free market ideology, latching on to a few superficial points that supported that reading while ignoring the vast majority of the actual content.

Most readers should be familiar with this sort of thing. There have been countless pieces in recent years detailing how people will warp reality however they need to to match their politics. This is usually attributed to hopeless ideologues who have shut themselves off into echo chambers where any new information is rewritten to support their team. From the ratified heights of the Economist’s editorial board to the fetid sewers of 4chan, this dynamic permeates the modern political landscape.

In contrast to this are the data wonks, typified by people like Nate Silver and Harry Enten. They care for analytical rigor for its own sake and they’re indifferent to the political implications of their analysis. They may have biases, sure, but they’re professional enough to not let them influence their work too much.

But the truth is these analysts are often just as prone to giving bad analysis as anyone else. And I don’t mean that their models seemed to predict one thing then another thing happened, that’s just probability. I mean they were making consistently bad predictions based on questionable assumptions. This isn’t because they weren’t using data and empirical research. On the contrary, they were often focused on the numbers to the point that they weren’t thinking critically about the broader framework they were using to interpret them.

But the effect was the same as it would be with any hard core ideologue: every new piece of information gets bent to the predetermined conclusion. They weren’t being biased, per se, but they clearly had expectations and were invested in them being verified.

Next time you read a piece of election analysis, the following questions. Do they seem to be presenting a causal relationship in a very shallow way? Are they clearly ignoring other potentially relevant factors? Can you realistically use the same data they are to justify a vastly different conclusion? Are they expressing a lot of certainty in a particular result even if the numbers aren’t really showing it? You’d be surprised how frequently analysts fail on these points.

The Great 2016 Screw-up

We could see a lot of this happen in 2015 and 2016 when analysts constantly seemed to expect that the Republican primary would eventually turn against Trump. They pulled out one reason after another to just this belief: the party was against him and the party usually decides, early favorites often fade, late deciding voters would pick a more serious candidate and so forth.

These were possibilities, of course, but they were being used to obfuscate the far more relevant fact that Trump’s lead in the polls was large and persistent. They were repeated with a high degree of certainty long after a point when it should have been clear they weren’t being borne out.

At points, many of these assertions didn’t even make sense, or at least were very questionable. For example, it was possible that that late deciders would want a more “serious” candidate than Trump, but it was equally, if not more plausible they were the sort of low information voters who’d go for Trump on the basis of simple name recognition. It was possible that terrorist attacks might make Republican voters look for a stable pair of hands, but they also could have pushed them to the candidate who played to their base xenophobia. Analysts consistently chose to interpret the data in a way that broke against Trump, without questioning why, and they were wrong every time.

I don’t think they did this because they biased or particularly cared whether Trump won or not. Instead I think they just thought Trump was such an aberration from what they considered to be normal. Therefore, they expected some sort of reversion to the mean to happen even. But that was the wrong assumption to make, and after some point that should have been obvious.

Bad Signs for 2020

It would be nice if analysts had learned their lesson from 2016. Unfortunately, early analysis of the 2020 primary doesn’t exactly inspire confidence.

Going by current polling alone, you’d have to concede that Biden and Sanders are obviously in the best position of anyone, polling well ahead of the other candidates. By contrast, other candidates like Kamala Harris, Corey Booker, Beto O’Rourke and others have struggled to gain any traction, rarely breaking into double digits polling-wise.

But until fairly recently analysts like Silver, Enten and Cillizza (who is considered and analyst for some reason), as well as betting markets, were basically ignoring all this and predicting the exact opposite of what the polling was suggesting. Sanders and Biden tended to be ranked fairly low in terms of their chances of winning the Democratic nomination behind candidates like Harris, Booker, O’Rourke, Gillibrand and others whom they’re currently beating by wide margins.

Biden and Sanders do have weaknesses, Biden in particular has a habit of blowing up Presidential runs pretty quickly. But they also have a lot of advantages. Biden doubtlessly has a lot of institutional leverage, and he may have enough momentum to brush past the sort of gaffes and scandals that would have derailed him in the past. Sanders has a dedicated support base, a strong campaigning infrastructure and has proven himself able to adjust his already popular platform in a way that makes up for past weaknesses without appearing insincere or ideologically inconsistent. Clearly, they shouldn’t be taken lightly, but they frequently are.

The dismissive tone is usually justified with the observation that early leaders often fade. Once the race begins in earnest, a lot of factors lying just below the surface of the candidates will play to the advantage of candidates like Harris, Booker, O’Rourke and others. They offer a couple of different rationales for why they expect this to happen, but the trouble is none of them are good ones.

Voter’s Second Choice

One factor that Fivethirtyeight pointed to was voter’s second choices. Voters change their minds in primaries, often quite a lot. Once the advantage of name recognition fades or the the current front runners collapse their support will start to shake out to other candidates.

But the trouble with this is that, according to a Morning Consult poll they cite, the top alternative for most supporters of Biden and Sanders are Biden and Sanders. That’s true for the supporters of most other candidates too. If one of their campaigns collapsed, that would actually seem to seal the race for the other. They’d both have to collapse for the race to be thrown open, or at least that’s what current polling suggests.

Demographics

Analysts tend to assume that election will break down largely across demographic lines, and they presume to know exactly how it will happen.  For example, an analysis of candidate strength Fivethirtyeight did, 3 of the 5 metrics were related to age or ethnicity.

Now, there are a lot of different things you can try to anticipate such support. You can look at how well candidates reflect the priorities of those groups. You can consider how campaigns work through the various trust networks in different communities. But more often than not, at this point analysts have a tendency to reducing the appeal of candidates to affinity bias. It’s presumed Harris will do well with black voters regardless of the fact that her record on things like criminal justice reform is out of sync with things like Black Lives Matter.  O’Rourke will do well with millennials because he’s a hip, youngish dude, even though his positions are usually quite a bit more conservative than the typical millennial’s. Yes, people generally do want to vote for people who they feel a sense of shared experience with, but there are other things too. In any event, analysts shouldn’t be making such strong presumptions absent on any actual polling data.

But where there is polling data the predictions haven’t panned out. The same Morning Consult poll cited earlier also showed that the front runners Biden and Sanders tended to be strong across the board, and there wasn’t really all that much variation across demographic profiles. Where there was it often cut against the narrative. Sanders, for example, was actually stronger among non-white voters and women, totally contrary to the conventional wisdom.

Endorsements

Another factor analysts will frequently point to are endorsements. There are a lot of interesting ways you can think about endorsements. For example, it can be an early indicator of how well they are likely to perform with certain groups long term since you can make good inroads into a community if you can wrangle the support of it’s prominent figures. Or it may indicate a candidate’s outreach strategy or base of support.

For example, it’s interesting that Kamala Harris and Corey Booker have locked up support in their home states while Bernie Sanders has gotten support that’s more shallow but broadly spread. That says a lot about where these candidates draw their strength, with Harris and Booker capitalizing off their years in state politics while Sanders is drawing strength from a somewhat more nebulous nationwide movement.

But analysts tend to reduce the importance of endorsements into a “party decides” narrative, where the party brass lines up behind one candidate and that support trickles down into support among the rank and file. This is certainly something that can happen, but it seems to be oversimplifying quite a bit, especially in a context where large contingents of voters are wary of party elites. A more nuanced perspective seems warranted.

Moreover, at the end of the day none of that matters if the endorsements aren’t having a tangible effect on the polls. Cory Booker may be winning the endorsement race at the moment, but it’s doing him no good. Candidates can get tons of endorsements, but that’s no guarantee that it’s going to help them get any traction.

Electability

Finally there’s electability. Media outlets have been keen to point out that voters rank electability very highly as a factor in their decisions, especially this year with Democrats so focused on beating Trump at all costs. This, it’s reasoned, will play to the advantage of more moderate, “mainstream” candidates who will be a safer bet.

The trouble with this, as many have pointed out, is that electability is a squishy, circular concept.  In reality the only thing that makes a candidate electable is often little more than being perceived to be electable. Aside from that, nobody even really knows what makes a candidate more electable, or even what voters consider to be electable. In typical parlance, “electable” is usually shorthand for “moderate” and “less risky,” but it’s not clear that’s actually the case. “Electable” to average voter could just as easily mean “most popular,” “having the best platform,” or “having the greatest political ability.”

Moreover, the predicted effect doesn’t seem to be panning out the way people seem to be anticipating in terms of actual voter preference.

Conclusion

Bad analysis has consequences. Analysts generate narratives, intentionally or not, that become sort of self fulfilling prophecies. If a candidate is deemed to be less viable, then voters will see voting for them as a waste even if they really are the best choice for them. Alternately, the analysis may just be obfuscating a reality that will assertive itself regardless. In this case, people will be encouraged to adopt the wrong strategy.

We saw the consequences of this in 2016, when the unwillingness of analysts to seriously consider the possibility that Trump might win created a sense of complacency and establishment cluelessness that played to his advantage. In 2004, a narrative about candidate electability encouraged voters to tack towards moderation and consensus building when what they really needed was a candidate who was going to challenge Bush and speak to the increasingly left priorities of its base.

We can’t let ourselves repeat these mistakes on the basis of bad analysis.