Why do we watch sports? It's a simple question with a complicated answer. Sports are a huge entertainment business – the NFL alone generates at least $7 billion a year in television revenue – so it’s easy to lose sight of their essential absurdity. In essence, we are watching freakishly large humans in tight polyester outfits play with balls. They try to get these balls into cups, goals, baskets and end zones. It's a bizarre thing to get emotional about.

There's no shortage of social science that tries to pin down the appeal of sports. There's the tribal theory, and the mirror neurons cavort, and the patterning hypothesis, which argues that sports take advantage of our tendency to hallucinate patterns in the noise. (Slot machines are fun for the same reason.) All of these speculations are probably a little bit true.

But I'm most intrigued by the so-called talent-luck theory, which was first proposed by the UCSD psychologist Nicholas Christenfeld in 1996. (His short paper has only been cited a single time, but I think it’s a brilliant little conjecture.) Here's the model in short form: humans like watching feats of physical talent, but we still want to be surprised. As a result, the most successful sports (i.e., those on Sportscenter) have found a way to engineer an ideal balance of skill and randomness. Thanks to chance, the underdog (which is a polite way of saying the less talented team) still has a chance.

So what’s Christenfeld’s evidence? He relied on a popular statistical measure known as the split half reliability coefficient. The measure is often used when assessing the reliability – that is, the internal consistency – of a psychological test. Let’s say, for instance, that you’ve developed a new cognitive assessment designed for NFL quarterbacks. In order to measure the internal consistency of the test, you should randomly divide the questions into two groups. The split-half reliability is a measure of the correlation between the scores of the different groups, with higher correlations signaling higher test reliability. (The best tests are said to “hang together.”) In other words, if the quarterbacks performed equally well on both halves of the test, then the test is probably measuring something, even if we still don’t know what that something is.

Christenfeld realized that this common statistical tool could be used to assess the reliability of various professional sports, including baseball, hockey, soccer, basketball, football and rugby. He randomly divided each of their seasons in half and then computed their split-half reliability. To what extent did a team’s success in half of its games predict its success in the other half?

The first thing Christenfeld discovered is that different sports generate very different reliabilities on a per game basis. Baseball, for instance, has a single game reliability of 0.008. If that seems low, it’s because it is – the NBA is roughly eleven times more reliable on a per game basis than MLB. (Hockey is smack in the middle, while the NFL has the highest single game reliability rating of any major American sports league. Only rugby is more predictable.) When I tell Christenfeld that I’m impressed by the unpredictability of baseball, he notes that the randomness is rooted in the basic mechanics of the sport, as the difference between a triple down the line and a double play is often just a few millimeters on a bat. “There is also no partial credit in baseball,” he says. “A hitter doesn’t get partial credit for hitting the warning track.” The end result is that success in America’s game is an all-or-nothing proposition, which increases the noisiness of victory. (As Christenfeld notes, sports that are more reliable, such as football, do give partial credit for performance: “Football has field position,” he says. “Even if you don’t score, assembling a long drive still has benefits.”)

But this doesn’t mean baseball is all luck and noise. Instead, Christenfeld points out that randomness of a single baseball game is balanced out by the fact that the baseball regular season is 162 games long, or ten times longer than the football season. What’s more, Christenfeld found the same pattern in every sport he looked at, so that season length was always inversely related to reliability. “The sports whose single games reliably assess talent have short seasons, while those whose games are largely chance have long ones,” Christenfeld wrote in his Nature paper. “Thus these sports, differing enormously in their particulars, converge towards the same reliability in a season.” Christenfeld then goes on to argue that season length is not an “arbitrary product of historical, meteorological or other such constraints.” Rather, it is rooted in the desire of fans to witness a “proper mix of skill and chance.”

I find this paper fascinating for a few reasons. For starters, it clarifies the appeal of sports. Although sabermetricians have gotten far better at measuring various kinds of athletic talent, from DVOA to PER, the entertainment value of sports is inseparable from the fact that the talent of players is intentionally constrained by the rules of the game. “If sports were pure contests of skill, then they’d quickly become genetic tournaments,” Christenfeld says. “But that’s not much fun to watch.” As a result, the most successful sports have evolved rules to encourage what Christenfeld calls an “optimal level of discrepancy.”

This model also comes with practical consequences, helping us evaluate potential rule changes to a given game. More instant replay? That will increase reliability, which might be good for baseball, but bad for rugby. What about changing the requirements of women’s tennis, so that players have to win the same number of sets as men? “The data suggest that women’s tennis is more reliable” – the best players are more likely to win – “so I’d guess that adding another set would make it too reliable,” Christenfeld says. Should we shorten the baseball season, as many fans and commentators have proposed? Since baseball already has the lowest season-length reliability of any major sports league, that’s probably not a good idea. “You never want the outcome to feel arbitrary,” Christenfeld says.

The NBA is probably the sport most in need of Christenfeld’s advice. According to his data, the season reliability of basketball is 0.890, which is far higher than the NFL’s season reliability of 0.681. Such reliability manifests itself as a competitive imbalance, as the best teams routinely dominate their lesser opponents. While the imbalance of the NBA is caused, at least in part, by "the short supply of tall people" - that, at least, is the conclusion of a 2005 paper led by the economist David Berri - these human factors are exacerbated by the league rules. “I think it’s pretty clear that the second half of the [NBA] season should be shorter,” Christenfeld says. “The history of basketball is the history of basketball dynasties. There are way too many games where the outcome is predictable.”

And then there is the larger lesson of Christenfeld’s research, which concerns the difficulty of managing the competing claims of talent and equality. If talent is fairly rewarded – i.e., LeBron James gets paid what he deserves – then inequality increases and NBA underdogs are even less likely win. To deal with this problem, most sports leagues impose salary caps on their teams, as they attempt to shrink the gap between the best and the worst, the richest and the poorest. Such parity makes the sport less predictable and more exciting; LeBron is underpaid for the good of the game.

In real life, of course, we’re not concerned about upsets and underdogs – we care about social mobility. We don’t seriously consider salary caps – we talk about marginal tax rates. Nevertheless, the basic tensions remain the same. While we want our society to be relatively reliable – every “game” should be a measurement of skill – we also don’t want a perfect meritocracy, for that creates a level of inequality that feels unfair. It’s also de-motivating, and can create a feedback loop in which the “underdogs” are even less likely to compete in the first place. If talent always win, there’s no reason to play.

Christenfeld, Nicholas. "What makes a good sport." Nature 383.6602 (1996): 662-662.