Pity the Cleveland Browns fan. Seemingly every year, the poor performance of the team leads to a high first-round pick: in this year’s draft, the Browns are making the first selection. And every year the team squanders the high pick, either by trading down and missing a superstar (Julio Jones in 2013) or trading up for a pick that didn’t pan out (Johnny Manziel in 2014, Trent Richardson in 2012, Brady Quinn in 2007, et al.) The draft is supposed to be a source of hope, a consolation prize for all the failures of the past. But for the hapless Browns, it has become yet another reminder of their chronic struggles.

This blog is not another critique of a pitiful team. The Browns might have a terrible track record in the draft, but I’m here to tell you that it’s not their fault. And that’s for a simple reason: picking college players is largely a crapshoot, a game of dice played with young athletes. The Browns might not know how to identify the college players with the most potential, but there’s little evidence that anybody else does, either.

It’s not for lack of trying. Every year, professional football teams invest a huge amount of time and effort into choosing which college players to take with their draft picks. This is for the obvious reason: picks are extremely valuable. (Because the NFL has a strict cap on rookie salaries, new players are significantly underpaid, at least compared to their veteran colleagues.) Given the high stakes involved, it seems reasonable to assume that teams would have developed effective methods of identifying those players most likely to succeed in the pros.

But they haven’t. That, at least, is the conclusion of a 2013 analysis of the NFL draft by Cade Massey and Richard Thaler. Consider one of their damning pieces of evidence, which involves the likelihood that a given player performs better in the NFL than the next player chosen in the draft at his position. As Massey and Thaler note, this is the practical question that teams continually face in the draft, as they debate the advantages of trading up to acquire a specific athlete.

Unfortunately, there is virtually no evidence that teams know what they’re doing: only 52 percent of picks outperform those players chosen next at the same position. “Across all rounds, all positions, all years, the chance that a player proves to be better than the next-best alternative is only slightly better than a coin flip,” write the economists. Or consider this statistic, which should strike fear into the heart of every NFL general manager: over their first five years in the league, draft picks from the first round have more seasons with zero starts (15.3 percent) than seasons that end with a selection to the Pro Bowl (12.8 percent). While draft order is roughly correlated with talent – players taken early tend to have better professional careers – Massey writes in an email that he “considers differences between team performance in the draft to be, effectively, all chance.” The Browns aren’t stupid, just unlucky.

If teams admitted their ignorance, they could adjust their strategy accordingly. They could discount their scouting analysis and remember that college performance is only weakly correlated with NFL output. They might even explore new player assessment strategies, as the old ones don't seem to work very well.

Alas, teams routinely act as if they can identify the best players, which is what leads them to trade-up for more valuable picks. But this is precisely the wrong approach. As proof, Massey and Thaler compute a statistic they call “surplus value,” which reflects the worth of a player’s performance (as calculated by the pay scale of NFL veterans) minus his actual compensation. “If picks are valued by the surplus they produce, then the first pick in the first round is the worst pick in the round, not the best,” write the economists. “In paying a steep price to trade up, teams are paying a lot to acquire a pick that is worth less than the ones they are giving up.”

Why are most NFL teams so bad at the draft? The main culprit is what Massey and Thaler refer to as “overconfidence exacerbated by information.” Teams assume their judgments about prospective players are more accurate than they are, especially when they amass large amounts of data and analytics. What they fail to realize is that much of this information isn’t predictive, and that it’s almost certainly framed by the same biases and blind spots that limit our assessments of other people in everyday life. As Massey and Thaler write: “The problem is not that future performance is difficult to predict, but that decision makers do not appreciate how difficult it is.”

There is something deeply sobering about the limits of draft intelligence among NFL teams. These are athletes, after all, whose performance has been measured by a dizzying array of advanced stats; they have been scouted for years and run through a gauntlet of psychological and physical assessments. (As the economists write, “football teams almost certainly are in a better position to predict performance than most employers choosing workers.”) However, even in this rarified domain, the mystery of human beings still dominates. We live in the age of big data and sabermetrics, which means that it’s harder than ever to know what we don’t. But this paper is an important reminder that such meta-knowledge is essential—when we ignore the error bars, we’re much more likely to make a very big mistake.

Bill Belichick, the coach of the New England Patriots (and former coach of the Cleveland Browns!), has won lots of games by pushing back against the curse of overconfidence. If Belichick has a signature move in the draft, it’s trading down, swapping a high pick for multiple less valuable ones. (Under Belichick, the Patriots have gained more than 25 compensatory draft picks.) If teams could reliably assess talent, this strategy would make little sense, since it would mean giving up on superstars. However, given the near impossibility of predicting elite player performance, gaining more picks is an astute move. Since nobody knows who to choose, the only way to play is to make a lot of bets.

Massey, Cade, and Richard H. Thaler. "The loser's curse: Decision making and market efficiency in the National Football League draft." Management Science 59.7 (2013): 1479-1495.