An Outsiders Inside view of the NHL Stats Revolution – Part 2


(In part 1 I discussed the current state of advanced stats in the NHL with a view to defining an “ideal state” for NHL clubs in their efforts to establish modern analytics departments. In part 2, we look at where this form of analysis came from and where it may be headed in the future)

“I’ve never said, never thought, that it was better
to be an outsider than it was to be an insider, that my view of the
game was better than anyone else’s. It’s different; better in some ways,
worse in some ways. What I have said is, since we are
outsiders…let us use our position as outsiders to what advantage we can.
Let us back off from the trees, look at the forest as a whole, and see
what we can learn from that.”

Advertisement - Continue Reading Below

– Bill James

Having been an early adopter and advocate of possession-based analysis, perhaps the most common complaint I encountered over the years was how, if corsi was so valuable, it was not actively employed by those who make their living inside the game. If the virtues of this analysis are so clear, why didn’t the experts come up with it? How could a bunch of no-name amateurs create something that could be of value to experienced, lifelong hockey men? 

It’s a reasonable question. The counter-intuitive truth of the matter is that hobbyists and outsiders enjoy distinct advantages that makes such advances more probable outside of the NHL’s official sphere. I’ve written previously about the potential value of the outsider perspective, as well as the structural obstacles that dissuade this sort of internal innovation but a more fulsome explication is perhaps in order.

Advertisement - Continue Reading Below

Top Down Vs Bottom Up  

Although teams are adding all sorts of adjunct layers to their front offices these days, most teams still operate as a top-down hierarchy, with a rigid, fixed power structure. The league is also notoriously insular, with the vast majority of executives coming from the ranks of ex-players, family members of past decision makers and lawyers/player agents. In addition, the amount of open positions in a system with a given number of teams is also more or less fixed. The result is the NHL as a business is less likely to experience new, disruptive models that challenge established ways of thinking.

As a result, there is a distinct lack of intellectual diversity. In fact, there are disincentives to stepping too far outside of the box. Just about everyone who rises to power in the league has been steeped in the same “hockey culture” for decades. There is an implicit antipathy towards things that don’t conform to NHL norms, and therefore an inherent risk to being “too different” for those whose career aspirations lie within league walls. Although there is significant attrition and relatively low career security in NHL position from coaches all the way up to the GM chair, there is also significant churn and intra-team recycling done within the confines of the NHL. Meaning one can fail, but fail successfully (ie; retain legitimate employment options) by not colouring too far outside the lines. It’s one thing to lose by perfectly conventional means. It’s quite another to fail while being stigmatized as odd by the rest of the league.

There is also a distinct difference in information flow between NHL teams and the sort of de facto, crowd-sourced peer review that produced corsi stats. NHL clubs are separate and disconnected, particularly when it comes to chasing strategic insight that will confer competitive advantages. As such, any particular insights that are gleaned from work inside individual franchises are horded and protected as state secrets. In effect, NHL teams are the proverbial collection of blind men trying to describe the elephant by feeling a single portion of the animal: they each have bits of information that are only portions of the whole.

In contrast, it’s much easier for amateurs to work on big picture innovation outside of the hierarchies and incentive structures within the league. Possession theory grew not only from the key insights of a handful of pioneers, but due to the rapid sharing and subsequent iterations engendered by digital self publishing and social media. Blogs and the like allowed an ever growing collection of amateurs to bring their widely diverse range of skills and abilities to bear on the issue of better understanding hockey. What’s more,  they were never subject to the pressures like those that exist in the league such as being fired or ostracized. The outsiders could focus on the big picture problems season after season and share their research freely, unencumbered by the tethers that would traditionally bind would-be innovators inside the NHL. 

The Times Are Changing

The issue of new digital self publishing raises the point that the technology, tools and, indeed, the game itself, have undergone fairly rapid changes over the last decade or so. A vast majority of the NHL’s decision makers grew up in an environment drastically different from the one we’ve seen emerge in the cap era: a league of increasing parity, complex budgeting demands and huge amounts of readily available statistical data. The evolution of the intellectual demands placed on coaches and GMs, as well as the skills and tools required to meet those demands, have likely evolved faster than the men in authority.

Furthermore, it’s a fair inference that the general mistrust towards stats one sees from many front offices isn’t merely a knee-jerk reaction to the unfamiliar. A vast majority of the league’s existence, scouts, coaches and general managers learned to be deeply skeptical of the predictive power of numbers – at least, of the numbers that were available such as goals, assists and plus/minus.

Advertisement - Continue Reading Below

Though many GM’s still obviously weight certain numbers heavily in their decision making (like goal scoring, for instance), they also deeply respect players with well established resumes (ostensibly limiting the risk of not “knowing” what a guy is) and give considerable regard to non-tangible factors like personality, attitude, role etc.

Possession-based theory has also confirmed that there are significant gaps between things like conventional counting stats and a player’s true value on the ice. In absence of this knowledge, however, these gaps significantly  increase the risk of failure in player evaluation and prediction. Thus, the entirely adaptive heuristic that “numbers don’t tell you everything” evolved. 

Of course, decision makers attempted to fill those gaps with the data and mechanisms they had at the time, including scouting, interviews and background checks. A swath of primarily qualitative information aimed at divining a player’s true “core” self and if he had the sort of game style and personality traits that would ultimately translate to success in the NHL. The official formula is to mix those things together to varying degrees, weight a player’s recent performance heavily, and then add a dash of gut feeling and executive intuition to build rosters.  

And most of them did (and do) this quite well (with a few obvious exceptions). The question isn’t whether traditional hockey men are bad at their jobs, but whether there’s room to do them better.

Advertisement - Continue Reading Below

More and More Complexity

The enduring suspicion of stats gives rise to the other most common charge against advanced stats in hockey: that the game is too dynamic and too complex to effectively capture with numbers. In fact, the exact opposite is actually true. The complexity of the game increases the need for systematic data collection and analysis rather than invalidate it. 

With the the differences between teams and players narrowing due to parity, it is becoming ever more difficult to tease apart the signal from the noise in the NHL – even for highly trained, highly experienced professionals. Like two equally talented poker players facing each other, the random drop of the cards becomes more influential to the outcome than it would between two unequal opponents. As parity increases, so too does the sway of chance.

The human brain tends to be great at detecting patterns, but lousy at understanding which of them is a product of chance. We are, by nature, practically innumerate and blind to things like base rates and natural variance in big data sets. We mistake illusory correlations for causation and find narratives appealing and explanatory, even though they tend to be reductive and dependent on a a heaping dollop of the hindsight bias. In the face of increasing complexity of information, people tend to retreat to comforting cognitive paradigms such as convention, ritual and even superstition to re-establish a perception of control

The idea that hockey is dynamic and complex is a recognition of the difficulty of data gathering, but nevertheless an argument in favour of disciplined, systematic statistical analysis, not against it. When used correctly, “big data” can be harnessed to tease skill from circumstance and chance in ways that may be next to impossible through intuition or qualitative means alone.

The Way Forward

Corsi stats have evolved to the degree that they can likely improve NHL decision making in some general, broad brush ways right away. They fill some of those aforementioned conventional stat gaps and provide a mechanism for identifying and understanding the role of variance in the league.

As mentioned in the previous article, the next step is marrying possession-theory and statistical modeling with existing experience and knowledge at the NHL level. Specifically, teams have the opportunity and the wherewithal to leverage the work that has been done in the wild so far and then apply their internal resources, scouts and video tools to yield new insights.

Deep data mining and granular analysis combined with video and observation can teach club’s about the skills and tactics that influence possession on both the individual and team level. By discovering what further moderates possession, teams could begin to develop robust models that predict a player’s future possession and goal differential based on his age, results and usage. They could also adjust their on-ice tactics to combat opposition’s strategies and build rosters. Most importantly, they can build internal processes to test their own hypothesis and assumptions while combating some of the counter-productive cognitive short-cuts that tend to skew rational decision making currently.   

The infiltration of new stats in the NHL has begun. What happens from here depends on how seriously teams take these new tools and how well they manage build their departments and integrate them into the decision making process at all levels of the organization.

  • First Name Unidentified

    This 2 part series is some of the best reads i’ve come across! Kudos to you Mr. Wilson

    I am a firm believer in utilizing stats and analytics for most critical decision making processes. Whether you are trying to invest in a new project or looking at a stock to invest in or trying to analyse amount and frequency of traffic flow over a given period of time.

    Similarly, while trying to pick players (or assets) I would certainly look at metrics that will answer a simple question – is it a good investment?

    I also believe that this metric system in hockey (or any other sports, for that matter) will be much more impactful if a metric such as “relative strength” is introduced which will compare the momentum of performance (or the velocity) of players in similar roles or positions (ie, centres, LW, RW or G). With this metric, we can also track how has a player’s statistics moved over a given time frame – up or down.

    I really do hope that the Flames adapt to this changing style in a hurry rather than just go with gut feeling.

  • Derzie

    There is a book entitled “The Dynamics of Software Development’ by Jim McCarthy. It is written by an ex-Microsoft guy who led the build of key elements of some of their signature software. It is a tome that is written in the language of the experienced software professional but laden with the hope that ‘the suits’ in charge of the budgets, decisions and timelines read embrace the message that software may seem like voodoo and the timelines are like shooting stars but the process and the value it provides are undeniable. And that there are right and wrong ways to do it but success is a combination of art & science.

    Kent’s series reads to me in a similar way. We are seeing some hockey people that are willing to step out a little and listen to the nerdy-half. Social media has played an immeasurable role in this process. Which is probably why business leaders are still confused and resistant to adopting the right way to look at and build software. No way to deliver the message in a meaningful way since they sure as shootin dont understand Jims book (or Kents). Well written piece Kent.

  • PrairieStew

    “As a result, there is a distinct lack of intellectual diversity.”

    Kent, have you considered diplomacy as a career option ? This quote made me laugh, it really sums it up.

    My experience, working in many different places over the last 20 plus years and often working with a variety of community volunteers that there is in fact a hockey mindset. Hockey “guys” are remarkably similar to law enforcement officials in that they see the world in very black and white terms. They know what is (right and wrong), but don’t spend alot of time thinking about why or how something is. One of the most telling examples I encountered as a small town Rec Director was the notion that all teams (from Novice on up) needed to have full ice practices. Coaches were concerned if they had “half ice ” practice – then their team’s skating skills would suffer. Sorry, the 8 year old doesn’t need to go 200 feet, he needs to learn to stop, start and turn ! Never mind the fact that if they shared their ice – they could practice more often !

    Don Cherry is an overblown caricature of the hockey mindset, but not as overblown as you might think. Each hockey guy has his view, and although they don’t always agree, they are similar in this: they are not interested in changing their views. “Bring all the evidence (Talk) you want – I am not changing my mind.”

  • SydScout

    Not sure why some people are so polarised when it comes to analytics.You’re either all in or all out.

    I highly doubt there are any teams that are not using analytics to some degree.Like any new idea it takes time to devolop some trust.

  • Its really not that surprising why “hockey men” didn’t figure out and develop more advanced stats. Most that I have met are meat heads who are trained to skate through the boards when told to and not question it. Of course there are lots of exceptions, but as a whole not the brightest stars in the night sky.

    • Robear

      While I might agree that there is a conformity of thought in the average hockey player (makes for a better team player) most every single player that I have met is very bright.

      Having said that, they do also tend to be meatheads (note to self: keep the daughter away from the hockey boys)

  • RealMcHockeyReturns

    Nice job as always Mr Wilson! Please don’t let your day job stop you from being a big contributor here. Just wanted to give kudos on the writing that does not stoop to conventional grade 9 reading levels. And for the great analogies like the blind-man-describing-elephant-by-grabbing-one-part (gee…what hangs lowest lol) and quite a number of witty analogies in part one.

    My take on the moves to modernize the NHL (or other sports) with advanced metrics is that sadly it begins with nothing-to-lose lower-tier teams to start the ball rolling (Buffalo similar to Oakland A’s in MLB), then slowly begins to spread until someone innovative, intelligent, and open-minded like Brendan Shanahan with Leafs to push it out to a more generally accepted level. Hopefully the NHL wakes up to video or other tools to help measure everything happening on the ice (like NBA’s SportVU system) and I agree with a commentor on part one who was encouraging Edmonton’s new arena designers or influencers to build in these “tools” into the arena.

  • The Last Big Bear

    I think the Nations Network editorial staff are massively underestimating NHL clubs and their understanding/utilization of ‘advanced’ stats.

    I have spoken to amateur scouts working in the CHL, who are using in-house scouting stats which I would consider to be well ahead of anything I’ve seen from mudcrutch or the like.

    I’ve been using advanced stats from sites like Behind the Net (and various sorely-missed sites which are now deceased or pay walled) to build my $20 hockey pools for like 6 years now. Everyone I’ve ever spoken to who works in the NHL in an off-ice capacity has a broad and sound understanding of ‘advanced’ stats.

    I find it very difficult to believe that companies with hundreds of millions of dollars on the line are ignoring this information, when they are clearly aware of its existence.

    Except the Oilers. The Oilers are a damned joke.

    • mattyc

      I’d like to think everyone is using/understands them (including their limitations), and yet we still see huge mistakes all the time. How does a team that understands possession theory buy out Grabovski, sign Andrew MacDonald or Derek Engelland (to name only a few)?

      • The Last Big Bear

        1) The Leafs are also a damn joke.

        2) What do you have against Andrew McDonald? He’s competent at the NHL level.

        3) I do not see Engelland’s signing as a mistake at all. There are few, if any, enforcers who are better at hockey than Engelland. His price tag is immaterial to a team at the cap floor.

        I’m too tired to really flesh out the rest of my argument; but it boils down to more of the same.

        Most decisions aren’t that bad because they’re looking at a different advanced stat than you are…

        And the Leafs and Oilers are incompetent.

        • mattyc

          Engelland is a relatively minor example. Andrew McDonald is a very poor possession player. It’s a little more nuanced than that (his neutral zone play is exceptionally poor), but I’m not sure I’d be shelling out $6 million longterm for that.

          It’s also worth distinguishing between ‘analytics’ and ‘possession theory’. As folks have emphasized, the quality of the data (and what you’re actually tracking) limits what you can actually conclude. I’m sure there’s lots of snake-oil salesmen selling all kinds of stuff that doesn’t necessarily have any proven predictive power.

        • Parallex

          “There are few, if any, enforcers who are better at hockey than Engelland.”

          That’s a low bar to set and even if it’s true (and for the record I don’t think Engelland is even the enforcer who is best at actual hockey on the Flames (from my viewings that would be Bollig) to say nothing of the league as a whole. You don’t want (or at least shouldn’t want) any enforcer logging the type of minutes required of a defenseman, that’s just asking for some bad juju.

          • Robear

            I might be attributing too much Machiavellian thought to BBBT, but my suspicion is that they didn’t want to get TOO much better while at the same being able to point to functional moves that provide some substance to the team.

            If I am right and BBBT are playing a deeper game, then I see them trying to do three things:

            1) Give the appearance of getting better for the average fan.
            2) Provide pseudo-openings for younger players by setting easy obstacles in their path, in the form of lower tier and aging players; and
            3) In the process, bag the 2014-15 season and ensure another high draft pick in 2015.

            Either that or they suck something fierce at player evaluation and we are in for a LOOOOOONG decade.

            Take yur pick 🙂

      • Burnward

        These stats are not infallible. We must remember that. They are a tool. Like the eye test, they also have their shortcomings.

        Those who can take the best of both will have the greatest success at evaluating talent.

  • mattyc

    I’d also like to extend my general appreciation for the Nations. The site(s) are a huge improvement (in both quality and content) on the generic conjecture that fills most of the mainstream sports media.

  • EugeneV

    The most important part of stat analysis would seem to me to be the actual stats collected. ie. the method and consistency of said collection of stats. The fact that 5 people may judge a scoring chance differently than another 5 people. Maybe they can collect scoring chance and SOG data like amateur boxing where 3 out of 5 stats collectors have to press a button within 1/2 a second of each other.

    The point I’m making is that the analysis will only be as good as the collection.

  • Gaudfather

    I’m curious would advanced stats have helped identify the SC winners in recent years? Has anyone seen an analysis of how the Kings, Hawks and Bruins advanced stats compared to others in the NHL.