THIS OCTOBER, THE statistically savvy Boston Red Sox will celebrate the 10th anniversary of their 2004 World Series championship. The occasion was as close as statheads have come to a man-on-the-moon moment, one where the advances became clear for all to see: One small step for David Ortiz, one giant leap for Bill James.
The Red Sox have since won two more World Series. But the real sign of progress for sports analysts is the increasingly ambitious questions they are asking. From studies on pitch framing to attempts to measure clubhouse chemistry, topics once thought to be beyond the realm of analytics -- the Dark Matter referred to throughout this issue -- now seem within reach.
That's how the party line goes, anyway. But before we celebrate the emergence of Analytics 2.0, it's worth reviewing how much influence the previous generation of analysts has had on America's three largest professional sports: baseball, basketball and football. Two of these sports show a clear commitment to analytics. The third reminds us that all the data in the world is for naught if it's not accompanied by a culture that empowers players, coaches and general managers to make better decisions.
The Red Sox, like Billy Beane's A's, were famous for emphasizing on-base percentage. They have been, in some sense, too successful for their own good. OBP is now the best predictor of free agent salaries (see: Shin-Soo Choo, seven years, $130 million), which means that while an appreciation for the statistic may be essential to building a championship contender, it is no longer a way to do so on the cheap.
Nor is number of walks an especially telling indicator of how much analytic progress baseball has made as a whole. Since a walk prevented is just as valuable as a walk drawn, teams have changed how they evaluate pitchers as well as hitters. In fact, the number of walks fell to 3.01 per team per game in 2013, the lowest since 1968, baseball's offense-starved equivalent of the Little Ice Age.
A clearer demonstration of how much analytics has changed the game comes from teams' elective choices -- options that opponents have little or no chance of preventing. Baseball analysts, for instance, have long decried the use of small-ball tactics -- the sacrifice, the intentional walk and the stolen base attempt -- which forfeit too much chance of a multirun inning to be worthwhile under most circumstances. Indeed, the number of stolen base attempts per team per game fell by 30 percent from 1993 to 2013, the number of sacrifice hits also by 30 percent (to their lowest per-game total since the statistic was first measured in 1894) and the number of intentional walks by 36 percent (also to an all-time low). All of this has come despite the recent renaissance of pitching and the revival of a lower-scoring environment, which would ordinarily be friendlier to one-run tactics. At least with respect to the sac hit and the stolen base, theory and practice are now fairly well-aligned.
If basketball analysts had a mascot, it would be either Shane Battier or his favorite shot, the corner three-pointer. The reason is clear enough: According to 82games.com, three-point shots from the corners produce an average of 118.8 points per 100 possessions. How impressive is that? It's roughly equivalent to a dunk or layup attempt from the restricted area -- or to Michael Jordan's career statistics (an offensive rating of 118.0 points per 100 possessions).
You might expect NBA teams to flock to such a shot -- and you'd be right. From the 1997-98 season, when the NBA moved back the three-point arc, to the current one (through Feb. 11), the number of corner-three attempts has increased from 2.34 per team per game to 5.48, a 134 percent increase. The rate of progress has tapered slightly in the past few seasons as defenses have begun to put as much emphasis on the shot as offenses. But the NBA has become a dunks-and-treys league, proving that stat geeks and NBA Jam had it right all along.
If there is a sport in which video-game strategy and analytical correctness are most aligned, it is football. Since at least 1971, statistical analyses have demonstrated that kicking and punting are not only no fun but are also done far too often. Teams ought to be going for it much more, especially in fourth-and-short situations at almost any time and any location on the field. Attempts to chronicle NFL teams' gutlessness on fourth down, from Bill Barnwell's Thank You for Not Coaching column to The New York Times' Twitter-happy 4th Down Bot, have become a pastime unto their own.
And yet, coaches and offensive coordinators have not changed their approach one bit. In 1991, NFL teams went for it on fourth down an average of 14.6 times per season, or slightly less than once per game. By 2013, the number of fourth-down attempts had increased to ... 14.8 per team. According to 4th Down Bot, NFL teams kicked or punted when they should have gone for it 693 times in the past regular season, or 21.7 mishandled situations per team. (By contrast, they went for it when they should have kicked or punted only 16 times -- not per team but in the league as a whole.)
Although 4th Down Bot's conclusions are based on long-term averages of NFL games since 2000 and do not account for factors like the strength of a team's short-yardage offense or the weather conditions, you'd expect those to even out over the long run. Instead, every team in the league exhibited a strong bias toward its placekicker and its punter, instead of its quarterback, tailback and offensive line.
How much difference does this make? I estimate, from 4th Down Bot's data, that the typical NFL team sacrificed about half a win over the course of the 16-game regular season due to its inferior fourth-down strategy. That might not sound like much. But 0.5 wins over a 16-game football season is equivalent to five wins over a 162-game baseball season. Baseball players who produce about five wins per season, like Choo, are borderline All-Stars who are offered $130 million contracts.
The reluctance of NFL teams to adopt analytic approaches is sometimes attributed to the limitations of statistics in the sport. It's true that there is much more Dark Matter in football than in baseball or basketball: Outside of quarterbacks and other skill-position players, we've barely begun to collect statistics on what the 22 players on the field are doing, let alone measure their worth. Fourth-down strategy, however, is one of the exceptions -- a relatively narrow and well-defined problem. To use a familiar analogy, it's quite hard to understand the morass that is American politics but quite easy to predict the results of upcoming elections by looking at the polls.
My view is that NFL coaches aren't irrational or necessarily ignorant of the statistics as much as they are poorly incentivized to get these decisions right. The average NFL team has been owned by the same family or organization since 1980 -- for the past 34 years. (By contrast, the average MLB and NBA team last changed owners in 1999.) Furthermore, because of the NFL's prodigious popularity and its generous revenue-sharing policies, even losing or incompetent owners possess extraordinarily valuable products. (The Jacksonville Jaguars are worth $840 million, according to Forbes.) This is a culture that fosters extreme risk aversion. Going for it on fourth down is risky twice over: in the micro sense of staking more on the result of one play, and in the macro sense of defying custom and tradition.
It was in baseball, instead -- the closest thing to a free market among the major American sports and one in which poorly run franchises often lose money -- where teams like the A's were forced to innovate. Moneyball was less an ideological commitment than a survival strategy.
Dark Matter discoveries have the potential to level the playing field among the sports, at least in theory. If 90 percent of what happens on a baseball diamond is adequately described by box score statistics but only 40 percent of what happens on a football field is captured, the latter sport has more opportunity to make gains through developments like motion-tracking cameras or more sophisticated study of the interactions among teammates. Moreover, the new areas of study are often much richer than Analytics 1.0. Being able to measure the break on Clayton Kershaw's curveball or the mileage Stephen Curry covers as he sprints up and down the court comes much closer to domains traditionally associated with scouting than statistics.
Human beings, however, are both the impetus for technological innovation and its greatest constraint. European soccer, which is as cutthroat economically as it is on the pitch, might benefit substantially from Analytics 2.0, as might the NHL, in which many franchises struggle to make a consistent profit. Baseball and the NBA should continue to make gains; they are already so competitive that teams might soon view analytics as less of a luxury and more of a necessity. But for the NFL, which has failed for more than 20 years to adopt some of the most basic tenets of analytics, Dark Matter is the equivalent of an iPad to a man who still uses a typewriter.
Follow The Mag on Twitter (@ESPNmag) and like us on Facebook.