If college football had a video game-style rating system, who would grade higher: Ohio State running back J.K. Dobbins or Alabama offensive lineman Alex Leatherwood?
How about LSU safety Grant Delpit vs. Oregon quarterback Justin Herbert?
Or Iowa defensive end A.J. Epenesa vs. Oklahoma center Creed Humphrey?
Starting today, we do have a rating system: the PlayStation Player Impact Rating. A 0-100 number for every FBS offensive and defensive football player.
For the record: it's Leatherwood (98) over Dobbins (91); Delpit (96) over Herbert (57); and Epenesa (94) over Humphrey (79).
How do we get these numbers?
While no human can watch every single player on every single snap every single week and objectively rate them all, a mathematical model built by ESPN Sports Analytics can.
We've got a full explanation below, but the abridged version is this: For every player, we compare his team's performance when he's on the field vs. when he's off it, and then we adjust for the skill of his teammates and opponents on every snap and consider whether each play was a run or a pass.
Think of it as real plus-minus, scaled 0-100, just for football.
To be clear, there are no traditional statistics built into our new rating. It quite simply tries to measure the effect of each player on every play. The advantage of this approach is that less obvious impacts from players -- a lockdown cornerback's presence on the field allows a safety to drift toward the box and augment the run stop -- can be captured by the model.
Football is an intricate game, and we're trying to measure the full impact of the players within it.
While the statistics behind the scenes are complicated, the output is not. The model produces a points above replacement for every player, and we translate that on a 0-100 scale relative to each player's position. That last part is important -- players are compared only against their own position. Using the example above: Herbert actually produced more value above replacement than Delpit last season, but Delpit was better compared to all other safeties than Herbert compared to all other quarterbacks. Got it?
Today we'll be talking about 2018 PlayStation Player Impact Ratings, but for players still in college football (and those playing for the first time this year), their ratings will update every week as we learn more about them.
One last thing: There's no Harvard-esque GPA inflation going on here. When we say the rating uses a 0-100 scale, we mean a full 0-100 scale. There are plenty of 11s out there.
But forget the 11s. We know what you're thinking: who are the 99s?
The 99 club
Not only was Kyler Murray the Heisman Trophy winner, the first overall selection in the NFL draft and a 99 in the PlayStation Player Impact Rating, but he actually had the highest rating of any player in college football in 2018. That means that not only was Murray the most impactful player in college football last season, but he also was better at quarterback than any other player was at their respective position.
He also isn't the only superstar with a 99 rating. Some notables with the de facto top grade (achieving a 100 is virtually impossible) included Alabama QB Tua Tagovailoa and WR Jerry Jeudy, and Clemson WR Justyn Ross and DT Christian Wilkins.
But the benefit of a metric such as this isn't just to tell us about the superstars we know: it highlights some other players that might not be considered quite up to the same standard, such as a trio of Georgia Bulldogs it regarded that highly in OT Isaiah Wilson, OLB D'Andre Walker and TE Isaac Nauta. Wilson will be back for the Bulldogs but Walker and Nauta were late-round selections in this year's draft.
A few complete stunner names also populated the 99 group, including Utah State OT Sean Taylor and Houston WR Courtney Lark. But if we're going to discuss why a surprise might pop up here and there, we might as well do it with ...
The quarterbacks
Kyler? Of course.
Tua? No problem.
Trevor Law-- wait, Zac Thomas?!
In the list of shocking conclusions the rating system came produced, Thomas finishing among college football's elite signal-callers with a 95 certainly is up there. So what the heck is going on here?
In general for these ratings, the explanation of "why" can often be found in either a team's on/off splits with a certain player or the teammate/opponent adjustments. In Thomas' case, it's both.
Appalachian State averaged 8.1 yards per pass attempt when Thomas was on the field last season, well above the 6.4 yards per pass attempt when he was off it. Not only that, but the Mountaineers also ran better when he was in the game (6.0 yards with compared to 5.3 yards without). That second part might partially be because of Thomas' own rushing ability, but also threat of his rushing ability coupled with the threat of his passing ability.
Then there's the teammates. Though Appalachian State had a slightly above FBS average offense last season in terms of efficiency, the Mountaineers had only two non-Thomas offensive players above a 70 (and three above a 60) in PlayStation Player Impact Rating. Despite most of the team's best players being on defense, Thomas managed to put up strong traditional statistics (he finished 13th in Total QBR) with little offensive help around him. That adjustment is crucial for trying to understand a player's true value.
(Regarding Mike Glass III, note that this is a rate stat, meaning he is judged on a per-play basis, so Glass is not punished for having significantly less playing time last season than the other nine players on the list above.)
Best of the rest
Who are the best returning players for the 2019 season?
Keep in mind that PlayStation Player Impact Rating doesn't measure who is the most athletic or who will be selected highest in the NFL draft -- though there sometimes is an overlap -- but rather who had the strongest positive effect on the game.
Some quick thoughts on a few of these players:
Zack Moss: Though he caught only eight passes last season, Utah was much better passing when Moss was on the field, suggesting his pass-blocking might have contributed to the Utes' success through the air.
Jevon Holland: Opponents averaged 6.1 yards per pass attempt when Holland was on the field ... and 9.1 when he was off it.
Gage Cervenka: Not only was Clemson better when Cervenka was in the game, but that held even if we control for quarterback and running back. When Trevor Lawrence and Travis Etienne were in the game but Cervenka was not, Clemson averaged 5.7 and 6.2 yards per attempt and carry, respectively. Those numbers jumped to 9.1 and 8.5, respectively, when all three players were in the game.
Teams at the top
The PlayStation Player Impact Rating can also produce unit rankings by team. Three of the four playoff teams ranked in the top four in offensive line unit ratings, for example (Notre Dame finished 19th).
Both teams from the national championship game also ranked in the top five in defensive line unit rating. What's Cincinnati doing up there? The model loved Bearcats' defensive tackles Marquise Copeland (94) and Cortez Broughton (89).
The NFL draft
We explicitly want to note that this rating does not project where a player will be selected in the NFL draft or how he will perform on Sundays. There are elements -- athleticism, growth potential, etc. -- of the draft that are simply not captured by our new tool. But that doesn't mean it's not interesting or useful to look at in relation to the draft. We can think of it something like a production grade -- worth noting but not, for draft purposes, the whole story. Here are the ratings for every first-round pick in last year's draft.
Twenty-three of the 30 players with a grade (Nick Bosa didn't play enough snaps to qualify and Tytus Howard wasn't in the FBS) recorded a rating of 80 or higher.
How the PlayStation Player Impact Rating is calculated
The base of PlayStation Player Impact Rating is an adjusted plus-minus metric, calculated by determining how successful a team is on every play in every game.
In Division I college football there were about 4.5 scores (TD and FG) per team each game the last couple of years compared to roughly 50 in the NBA (FGM and FTM). A "plus-minus" based metric requires lots of scoring for proper evaluation.
To compensate for that we use a metric called expected points added (EPA), which assigns a point value to the change in field position a team makes each play in conjunction with the down and distance. Essentially, EPA knows that a 5-yard completion on third-and-3 is good, and a 5-yard completion on third-and-8 is not as good.
With EPA established, we need to know who is on the field and off the field on each play. Armed with that information, the Bayesian hierarchical statistical model then estimates the value of each player based on the EPA when he is on the field vs. when he is not, adjusting for the skill of other 21 players sharing the field on each play.
This part is just like ESPN's real plus-minus, though we can't stop there due to some differences between basketball and football.
One important feature to this model that is unique to our approach is allowing the model to determine how much importance each position holds by measuring the variance of player impact across all players with a common position.
The model estimates that top quarterbacks have more value because there is a larger spread in impact among QBs than, for example, tight ends. This feature is key as it allows the model to distinguish the impact between a quarterback and -- for example -- his center, as the two might often be on the field at the same time.
Additionally, the model recognizes the difference between run plays and pass plays. In general, pass plays are more valuable than runs. But consider the situation of an exceptional run-blocking fullback who plays much more often on runs than passes. If we compared his on/off splits to all plays, it might appear as if he is not valuable, because the plays he is not in on are more pass-heavy. But in fact, he does have value, because he improves the running game. To compensate, we adjust for run versus pass. (Note: Pass plays are compared to run plays, but an adjustment is made.)
Once every player's adjusted plus minus per play is established, we translate that number into a 0-100 rating relative to other players' adjusted plus-minus at the same position. That 0-100 number is the PlayStation Player Impact Rating.