Wysh List: The most overrated advanced stats in hockey

Ron Hainsey was a plus-30 last season.

That means Ron Hainsey was sixth in the NHL in something, at a time when Ron Hainsey shouldn't be sixth in the NHL in anything. Go back 15 years, and a plus-30 would have been an indisputable measure of a defenseman's prowess. But we've gotten a lot smarter and savvier as hockey fans in that span. We know that the Toronto Maple Leafs gave up more shot attempts and shots on goal than they had themselves when Hainsey was on the ice at 5-on-5. And we know that, relative to his teammates, Hainsey's performance was inferior.

This isn't meant to pick on Ron Hainsey. It's meant to illustrate that advanced stats and analytics have helped expose and supersede the lazy junk numbers we relied on for decades. Well, that and a universal understanding that if a player is skating to the bench when a goal is scored by his teammate, he's a "plus-1" despite having nothing to do with the scoring play. Which is nuts!

For that reason, there's been a tendency to deify analytics. But even fancy stats have their ugly numbers. And the wonderful thing about the analytics community is that everyone believes that their kung fu is the best. One person's garbage stat is another person's gospel.

This week, I wanted to try to define which widely accepted analytics are seen as abhorrent as "plus/minus" by the community that developed them. So I'm turning the column over to a collection of stats analysts, hockey geeks and numbers-crunchers to give me the banes of their analytic existence. (As many as I could: Sadly, some NHL teams wouldn't let their analytics departments play in my reindeer games.)

Here are the worst fancy stats in hockey, as presented by the great women and men of the hockey analytics community.

Also in this week's Wysh List: Jersey Foul | Puck headlines
Winners and losers of the week

Alison Lukan, The Athletic

There's one stat that will immediately make me cringe and that's "with or without you," or "WOWY."

The idea behind the measurement is sound; how do we measure "chemistry?" People wanted to find a way to look at how a player truly performs and/or how much his teammates may or may not be influencing those contributions. Hence "with or without you"; how does Player A do with Player B? How does Player A do without Player B?

What people end up doing is taking any stat and comparing what that measure looks like in those two scenarios; with each other, without each other. The conclusion often becomes if the measure is worse without Player B ... well, then Player B is obviously a key to Player A's success.

Here are the problems with that line of analysis. First, you're comparing just two players when there are three other skaters out there from the same team. What about those players' impacts or lack thereof? Second, sample size. Unless we have significant minutes in both the "with" and "without" categories, you're not really using relevant data. Third, to really try and isolate a player, you have to look at a ton of possible combinations: Player A with B, C, D, E; Player A with B, D, E, F; and so on. And you can't sum these outputs; each scenario is discrete. Fourth, WOWY neglects context. Why were players put together or separated? Score? Injuries? Matchups?

Finally, and arguably most importantly. the biggest problem with using WOWY is that there are so many better ways to determine what an individual player's impact is. WOWY made sense when it was the best measure we had, but today, we can use relative measures that consider how a player performs relative to his teammates; or even better, we can use tools like player isolates on Evolving Hockey's regularized adjusted plus-minus (RAPM), which gives a much clearer view of a player's individual value. WOWY may be fun to say, it may bring back memories of a great U2 album, but other than that, it's not really something we should be using today.

Travis Yost, TSN

The biggest reason plus-minus is such a useless measure is that it's not exclusive to any one game state. Plus-minus includes all even-strength goals, goals with goaltenders pulled, and even short-handed goals. In other words: It is impossible to use plus-minus to make apples-to-apples comparisons across players.

So the hockey community came up with a brilliantly simple way to fix plus-minus, and that's to use goal percentage. Goal percentage -- full disclosure, it's a measure I use to this day -- simply aggregates the goals scored by a player's team with him on the ice, and divides out by the total number of goals scored, both for and against. We tend to use goal percentage for even-strength and 5-on-5 comparisons, and for descriptive purposes, it's quite useful.

The problem is that people still, to this day, use goal percentage as a meaningful data point for forecasting or predicting future goal percentage, and that is outstandingly dangerous. We already know expected goals and/or Corsi for percentage are vastly superior measures at predicting future goal percentage, both because the sampling data is substantially larger and the impact of small sample variance are substantially lower. We also know that goal percentage is a very weak predictor of future goal percentage for a multitude of reasons.

Context is key.

Micah Blake McCurdy, HockeyViz

My least favorite widely available stat -- I don't know which stats are "advanced" -- is save percentage.

Every goalie sees a different profile of shot difficulty, and counting every save as if it required the same amount of skill to save, as save percentage implicitly does, is silly. Cutting the data down to try to mitigate this (like using only 5-on-5 save percentage, or 5-on-5 "high danger" save percentage) generally does more harm than good, because it's throwing out so much data. We'd never say that it was sensible to compare the test results of two students who took two totally different tests, but we don't seem to mind doing it with our most basic goaltending stat.

Shayna Goldman, Bench Bosses

For me, it's the absence of good advanced goaltending stats, which force us to rely on statistics like goals-against average and save percentage to describe a goaltender's abilities.

Goals-against average, in particular, lacks context. It doesn't detail the goaltender's workload in terms of shots or shot quality, which is essential when evaluating a netminder. For example, John Gibson had a 2.84 GAA last season, which can give the impression that he was worse than a number of goaltenders. He also had the highest expected goals against in the league of 180.06, according to Evolving Hockey, and unsurprisingly the highest goals saved above expectation (26.9).

The latter numbers give us a much more nuanced -- and accurate -- view of the player's true capabilities.

So, what should replace GAA and become the widely accepted goaltending stat? Well, that's the question a number of analysts are trying to answer.

Expected goals and goals saved above expectation are both useful in evaluation, and so is goals saved above average (GSAA). Another metric worth exploring is delta save percentage (dSV%), though that improves one commonly used goaltending statistic and not the other. Micah Blake McCurdy's work on competent goaltending evaluation that he presented at the Seattle Hockey Analytics Conference helps assesses goaltenders as well.

Relying on a statistic like GAA to evaluate goaltenders allows for misinterpretation of their play, and the fact that there isn't just one goaltending statistic that stands out as a replacement that's both digestible for the traditional thinkers and accepted by data-driven analysts hurts goaltending analysis as a whole.

Ryan Stimson, Author, "Tape To Space"

It has to be "with or without you" stats. They simply don't tell people what they think they do. There are myriad factors that go into smaller samples of performance (competition, other teammates, deployment, etc). And it's not just who is on the ice with them, but who isn't that is a big factor in WOWY being misleading. If a team is strong at a certain position, a player further down the lineup will inevitably have his WOWY data compared to those of stronger teammates with whom he doesn't get a chance to play.

Similarly, teams with only one good option at a position will see their numbers inflated. A regression-based approach will be able to account for each of these factors in a much more comprehensive and reliable way than splicing data. Many people on social media often draw the wrong conclusions using these stats.

Josh and Luke, Evolving Hockey

It's "with or without you."

WOWY analysis normally goes something like this: "Leon Draisaitl has performed significantly better with Connor McDavid than without him. Look at his WOWYs and you can see he is clearly carried by McDavid." What this type of analysis misses, among other things, is that a player doesn't just play with one other player. That player plays with a combination of all other players on their team -- four at a time, in fact, during the 5-on-5 strength state.

So, when Draisaitl was playing with McDavid, who else was he playing with? And when he was not playing with McDavid, who else was he playing with? It would stand to reason that the three players who were not McDavid would have a bigger overall impact than one McDavid.

WOWY focuses on a single pair of players and does not account for every other player who was on the ice. While this could be somewhat informative when looking strictly at line combination or player pair evaluation, this type of analysis is often used for the overall evaluation of a player.

Ideally, a metric that accounts for each teammate a given player played with while also accounting for the amount of time that player played with each teammate in aggregate is a far superior method. Relative to teammate metrics, RAPMs (regularized adjusted plus-minus, which is available on our site) or Micah Blake McCurdy's isolated player ratings account for all teammates in a much more comprehensive way.

Rachel Doerrie, "The Staff & Graph Podcast"

For the record, I think goals-against average is legitimately the worst stat around, and the fact it is used to decide value of goalies is a joke. It's a team stat. But that's not an "advanced stat." So, moving on ...

I don't take umbrage with a specific stat per se, but how specific stats are presented. Stats without qualification don't tell nearly enough of the story. A good example is Corsi for percentage. I see tweets from various people saying this team's Corsi is this or this player's Corsi is that, or the expected goals is this or that. To me, if it isn't relative to his teammates or the score-adjusted situation, you're missing a huge chunk of the story. It's the equivalent of telling a story without a setting.

A team that is losing 4-1 should be generating a huge portion of the expected goals or possession; they're down 4-1! On the flip side, most teams leading 4-1 would be playing a 1-2-2 by that point and sitting back to hold the lead. Of course they are going to give up more. That's the score-adjusted side of things, which is a key part of whether or not a team or player played well. If you're losing and still getting out-possessed and out-chanced, well, that's no good. But your coach likely changes the style once you hold a lead or are down a few, and that's a very important qualifier. With relative stats, how good are you compared to your teammates?

Connor McDavid is a perfect example. Edmonton got obliterated last season when he wasn't on the ice, and they were extremely dangerous when he was. Even further, it tells you how awful someone's linemates are if they are plus-7% in relative Corsi and their teammates are minus-2%. How is there that big of a discrepancy? It does no good to say that "a player's Corsi was this, so he's bad," when you haven't considered he probably played seven of the last 20 minutes while they were up a couple goals and the other team was pressing.

That's a long-winded way of saying that plain stats with no qualifiers do almost nothing for me. I want to know how you are relative to your teammates and what situations you were in. If you were 65% in possession when your team was leading, that's really good. If you were 50% while your team was losing ... that's not good enough. I think if you don't use the score-adjusted or the relative numbers, you won't get the entire picture.

Brian MacDonald, ESPN

I don't like Corsi because the name is not descriptive of what it is, and because it doesn't include shot locations. Both of these attributes hurt its ability to be accepted by non-analysts in the hockey community.

For years all people talked about on social media was Corsi and how awesome it is, and how shot locations don't need to be considered because they don't help with predictions. That was not only untrue -- because shot locations do, in fact, help with predictive ability -- but counting a blocked shot from the point as much as a shot on goal from the slot turned off a lot of hockey professionals who are analytics skeptics.

First impressions matter, and the first "advanced stat" the skeptics heard of was Corsi, and they thought it was ridiculous.

Sean Tierney, Director of Analytics, Hamilton Bulldogs (OHL)

While thinking about this question, I decided to create a chart (as I'm wont to do) in order to prepare an answer. I tweeted out the charts (I'm wont to do that, too) as a way to talk about my least favorite advanced hockey stat: zone starts. Why don't I like it? It isn't because the stat can't be valuable or that the stat doesn't convey some information. It does both of these things. The problem is that zone starts is misinterpreted, misunderstood, and, therefore, often rendered meaningless. To wit:

Coaches control zone starts, not players

First, zone starts are sometimes used to describe a player's ability. We've all heard of the shutdown defenseman who gets all the tough defensive zone starts or the offensive wizard who's always used in the offensive zone. But zone starts are not about a player's ability at all -- they're about a coaching staff's perception of that player's ability.

For example, Golden Knights star and defensive genius Mark Stone has started in the D-zone as often as Islanders forward Tom Kuhnhackl, who has been one of the NHL's worst defensive forwards in the season's early weeks.

Stone is a Selke Trophy candidate, while Kuhnhackl is fighting for his NHL life one game at a time. But their D-zone usage rates show that they're deployed similarly by their coaches. It's not about their ability but their coaches' perceptions of their value to their respective clubs.

While hockey broadcasts continue to be comfortable asserting that zone starts are the context we need to understand which players are good defensive (or offensive) players, this stat really only tells us which players are being used in which ways by coaches, regardless of that player's merit. This leads to my second point.

Zone starts don't matter (much)

Even understanding that zone starts are a coaching decision, not a player decision or ability, the real kicker with this advanced stat is the degree to which it matters. It's common enough to hear that poor shot metrics for a player can be understood and accepted if that player routinely receives tough starts -- think of all those fourth-line, Jay McClement types who take tough faceoffs in the defensive zone, post questionable possession metrics, but are given a pass because they do the dirty work of playing hard minutes to free up the offensive stars to enjoy the cushier offensive zone starts.

The problem here is that zone starts don't actually matter that much. That's because they're much more infrequent than commonly believed. Micah Blake McCurdy has shown that if we adjusted for defensive zone (or offensive zone) start-heavy players, even in extreme cases of deployment, only 5% of players would see a change of more than 1% in their Corsi for percentage. That's astounding. And small.

This is because, on average, players start approximately 60% of all their starts on the fly (i.e., not in any zone in particular) and another 15% of their starts in the neutral zone. On average, only 25% of a player's zone starts happen in the offensive or defensive zone combined. It's much, much tougher for a coach to control which players are heavily deployed in defensive or offensive zone situations than an observer might believe at first blush.

Say there's one minute left in the game, defending a 3-2 lead, with the crowd collectively holding its breath as the home team tries to eke out 60 seconds of play without a goal against to secure a season-defining win; in those cases, coaches will deploy the players they believe to be their best defensive options. In these cases, zone starts really do matter.

Any other time? Zone starts don't matter much.

Meghan Hall, HockeyGraphs

Two stats that always make me grit my teeth a little bit are power-play percentage and penalty-kill percentage for evaluating special teams.

For one, all the special-teams opportunities (i.e., the denominator) aren't necessarily the same length, which makes raw comparisons less useful. Also, the percentages don't allow for nuance in terms of when the goals were scored; power plays that score quickly should get more credit, and vice versa for penalty kills.

Instead, I like to look at the goals per 60 minutes rate or, if you want to remove the influence of the goalie, shots per 60 minutes or expected goals per 60 minutes. The percentages are fine as a single data point, but I see too many people use them as the be-all and end-all stat for determining the quality of special teams and how they rank in relation to each other.

Stephen Burtch, Sportsnet

My least favorite advanced stat that I see people make use of is any of the "relative" stats, which are simply the stat in question while the player is on the ice minus the same stat for their team with them off the ice.

The reason this bothers me is there is zero context employed in the use of "rel" stats. Basically, if a player primarily plays with weak linemates (or strong ones) while they are on the ice, and the rest of the team is stronger (or weaker) when they are on the bench, then their "rel" stats will look worse (or better).

I would prefer people use "relative to teammate" versions of these stats (also identified as RelT). The difference being that they are a more complex version of a team-wide average "with or without you" statistic, weighted by the amount of ice time the player in question plays with all of their teammates.

"Relative to teammate" stats are far more complicated to calculate than "relative" stats, but they are also far more meaningful and informative.

Dimitri Filipovic, ESPN

I'd say my biggest sticking point is the general application of context.

I think nearly every metric has at least some kind of utility if it's equipped with the right level of nuance, but you can't just pick and choose based on a narrative you're trying to push. I remember the early days of the analytics era in hockey, where every piece of writing -- mine included, by the way -- would make sure to cite a player's usage with regard to the quality of competition they were facing, and the number of offensive zone starts they were receiving. Both of those still have value because they're two pieces to the puzzle, but there's also so much more to it that needs to be considered. Who someone is playing against is undoubtedly important, but it doesn't paint the full picture unless you're also considering who that player is routinely playing with. The same is true for zone deployment, simply because so much of the game is played on the rush.

These aren't necessarily analytical by definition because they've been around forever and appear on basic box scores, but because they're both statistics that I still see spewed on hockey broadcasts and ones that are totally dependent on context, it's worth reiterating here: Faceoff percentage isn't nearly as important as you'd think, and giveaways are entirely circumstantial.

To illustrate that point, here are the five best players by faceoff percentage last season (minimum of 100 draws taken): Tomas Nosek, Patrik Berglund, Jason Spezza, Derek Ryan and Travis Zajac. Among the bottom 10 you have names like Evgeny Kuznetsov, Elias Pettersson and Mathew Barzal. The faceoff itself is just one play in a series of events, and it shouldn't be treated as more important than the rest just because it happens to come first.

As for giveaways, here are some names that were in the top 10 of players who turned the puck over the most: Johnny Gaudreau, Brent Burns, Leon Draisaitl and Drew Doughty. Here are the five players who turned it over the least (minimum of 20 games played): MacKenzie MacEachern, Cody McLeod, Markus Hannikainen, Zac Rinaldo, and Josh Currie.

If you constantly have the puck on your stick and are trying to make good things happen with it, you're generally going to wind up giving it away on occasion. As counterintuitive as that may sound, it's not a bad thing.

Jersey Fouls

From Mike Chambers of The Denver Post:

Two really pumped young fans here at #Avs AM skate. Nice family from Cole Harbour set up by 87 and 29 pic.twitter.com/rBoSuAdFyJ

— Mike Chambers (@MikeChambers) October 16, 2019

There's something to be said for the adorable community pride on display as two youngins from Cole Harbour celebrate their native sons Nathan MacKinnon and Sidney Crosby. It might fly as a FrankenJersey were it not for the fact that the Avalanche and Penguins were playing that night. No room for fence-sitters. Total Foul.

Winners and Losers of the Week

Winner: Dave Tippett

Of all the big-name coaches who landed with new teams last summer, Tippett was oddly overlooked as a game-changer. Perhaps because most of us assumed that the Edmonton Oilers would be hot garbage no matter who was behind the bench. But while Connor McDavid has 17 points in seven games (!), Leon Draisaitl has 15 and James Neal has eight goals, Tippett's system has the Oilers at eighth in the NHL in goals-against average (2.71) after they finished 25^th (3.30) last season.

Loser: John Hynes

The company line for the Devils is that Hynes requested that assistant general manager Tom Fitzgerald come down from the front office to behind the Devils' bench. Whatever the catalyst for it, it feels like a desperation move for a team that's gone six games without a win and blew a three-goal lead in the first game of a six-game homestand. Furthermore, it feels like that scene in "Office Space" where the Bobs come to the office to start evaluating each employee. "What would you say you do here exactly, John...?"