Random Thoughts – On Process vs Results

 

 

The recent struggles of Sean Monahan is a topic of interest for me, in part because it sparked a discussion about the nature of analysis, particularly regarding valuing results and processes. Let’s get to the Monahan stuff first and the rest will follow. I’ll talk about Mikael Backlund (yes, again) here as well.

Since returning from injury, the Flames rookie center has been getting absolutely killed. Not including last night, Monahan had been outshot (including blocks and missed shots) 48 to 15 or -33 over just three games at even strength. That’s a corsi % of 24%(!!). To put that ratio in context, the worst regular skater in the league by this measure so far this year is the Leafs Frazer McLaren at 36.6%. Brian McGrattan’s corsi% is 41.1% so far (worst forward on the Flames).

Keep in mind, this is while playing with Curtis Glencross and Jiri Hudler and getting fairly generous zone start bumps – the kid’s offensive zone to defensive zone faceoff ratio in those three games was 80%, 55.6% and 60%, respectively. So circumstances don’t explain him getting run over.

Even in such a small sample of games, these are worrying possession numbers and suggest something has gone terribly wrong. Either Monahan has hit the dreaded "rookie wall" or he has come back from injury too early. Either way, his night in Detroit began the same way, if you’re wondering why the kid was benched in the third period.

Anyways, my noting the Monahan’s dreadful outshooting numbers after the Rangers game (22 shots against, 6 for, -16 corsi) led to some discussion on Twitter. I was reminded that the Monahan line scored two goals despite being grossly outshot and that those are the only results that matter. Two goals for and zero goals against means a good night of work, all other considerations be damned.

Well, not really. As I note here continually, the best long-term predictor we have for outscoring in the NHL is outshooting (ie; positive possession). I think even the most innumerate amongst us can also recognize that getting outshot 22-6 but coming out with a 2-0 goal differential is more horseshoes than ability.

This brings me to the discussion of results vs process, which I’ll illustrate with a personal anecdote:

Years ago some friends and I used to get together monthly for a night of texas hold’em poker. This particular evening I was fortunate to be one of the last two players at the table. The game had gone on for a long time, it was a weeknight and my stack was beginning to dwindle. We also gave out a portion of the pot to the second placed guy, so with the guarantee of winnings I simply decided to throw the game and go home.

On the next hand, my two hold cards were a 2 and 9, off-suit. Perfect, I went all-in before the flop. My adversary called with something much stronger – a couple of face cards or something similar. He was much more likely to win, which was my intent anyways.

Except the flop happened to come up 9, 2 other. I had two pair and I went on to take the hand. I got a good result despite a terrible, low percentage play. It happens in cards.

And it happens in hockey. One of the reasons I was drawn to better, more predictive metrics in the early days of my own hockey writing is because I found myself too often chasing my tail in terms of my assessments of teams and players. I frequently found that the "numbers that matter" – ie the results like goals and wins – didn’t seem to accurately predict who was actually the better player or club down the road. I found myself bouncing around and flip-flopping, resigned to deploying shopworn, retroactive narratives like confidence and other pop psychology to explain away things when they didn’t accord with what had seemed to be solid conclusions.

Like a bad poker player who chases flushes, projects his betting habits and doesn’t understand how to bluff, bad hockey teams and players can nevertheless experience pockets of success. Sometimes you get the cards. Sometimes the puck bounces off a skate and goes in or your goalie stands on his head. Sometimes luck can favor you for longer than a game or hand. But, eventually, playing lousy catches up to you.

The objective in hockey is to score more goals than the other guy. When that happens, it can be seductive to assume that the game was played well. But it can be a false signal. Process matters. That’s why, despite winning big with a 9-2 off suit that one time, I still fold the hand whenever it pops up.

Which brings us to Mikael Backlund, whose recent spate of good play has ironically made him a player of contention in some quarters again.

Let’s first break down his last few games like we did with Monahan – in the last three, Backlund’s raw possession numbers have been +33 -28 for +4 or a ratio of 54%. This is especially noteworthy because the Flames have been soundly outshot in each of those contests – Backlund’s corsi rate relative to the rest of the team in those games was +15.7%, +32.9% and +13.3%. Unlike Monahan, Backlund wasn’t starting more shifts in the offensive zone either – his o-zone to d-zone faceoff rate was 22%, 23% and 45%, respectively.

So that’s some quality work and the reason why Mickis has lead the Flames forwards in ice time recently. You have to go back to the end of November to find the last time Backlund was scored on at even strength (against the Ducks), which is a function of good play and luck (100% ES SV%!). Still, this is Backlund’s under appreciated talent – the puck moves the right direction when he’s on the ice. He doesn’t do it by hammering opponents with big hits or making Datsyukian dekes through the opposition. It’s just quiet, dogged puck pursuit and an ability to read and react to the play. Which is why he’s an uninspiring player to many pundits and fans – we tend to notice and remember the profane and spectacular and not much else.

One of those pundits is Dean Molberg (Boomer) on the FAN960 in town. A recent exchange between him and friend of the program Ryan Pinder on the radio this past week sparked me to write this section…

The argument was over Backlund, with Pinder’s defense couched in possession metrics. Boomer, a staunch disbeliever in corsi, stated that whatever the numbers say, Backlund seems too passive and uninvolved to be a player of any value. The argument devolved into Molberg more or less telling Pinder to step away from the "nerd stats" and watch the game with the implication being that Backlund would fail the eye-test absent the crutch of arcane measures. 

Let’s first establish that the legitimacy or utility of evidence is not subject to any given person’s subjective appraisal or incredulity. Information is information, even if what it implies is personally hard to digest or fathom. Every major discovery or advancement of knowledge in human history was initally met with disdain, contempt and dismissal by people in general and established authorities in particular. Truth is immune to popularity.

Furthermore, let’s also establish that anyone who has to muster a smear or stigma to invalidate evidence is immediately behind in an argument. One of the easiest ways to rationalize away uncomfortable or counter-intuitive information is to label it as somehow heretical and therefore not worthy of serious contemplation.

An analogy…the theory of evolution is based on lots and lots of data and evidence. It’s the best understanding we have of how life grows, changes and persists on this planet. But there is a large swath of so-called "creationists" who dismiss evolution out of hand because its conclusions do not accord with religious dictum. So the evidence behind evolution is labeled blasphemous and redacted from the minds of true believers because it is rationalized as inherently wrong. 

My intent is not to call Dean the hockey version of a creationist (seriously…I’ve met Boomer and like him), but to point out how  cognitive dissonance can make people act and think in less than rational ways.

The irony is that suspicion of numbers is not unwarranted. As I mentioned above, people most certainly should be suspicious of the value of outcomes like wins, goals, points, etc. (particularly in small samples). One wonders if the established hockey orthodoxy grew distrustful of stats over time because so many of them (*ahem*plus/minus) didn’t paint an accurate picture of a team or player.

The problem is, no one treats their observations with a similar level of suspision. We’re hardwired to believe what we see and, further, to take subjective impressions and compelling anecdotes as implacable indicators of truth and reality. No matter how much you respect the power of a given metric or series of numbers, they never seem as viscerally real as the instant judgements and gut feel we get from watching tha action.

Which is why it’s sensible to respect and investigate quantitative evidence and admit, at times, that subjective evaluation might not completely line-up with reality. Humans are good at naturally detecting patterns in data, but we’re terrible at telling when those patterns represent real causal relationships or not.

I’ll finish by noting that the current disruptive "new stats" like corsi and PDO that give so many established MSMers (and some authorities in the league) fits weren’t constructed by theorists in a lab far away from the game. They were stumbled upon and tested in a series of iterative studies by passionate, invested hockey fans and analysts – a sort of crowd-sourced empiricism where the utility of possession metrics and percentages was tested and re-tested and then built upon when they proved to be relatively robust. To put it another way, these aren’t numbers cooked up by people who don’t watch hockey and are simply looking to completely up-end current conventions because "screw you jocks!"

For my part, I think the point of discussing and disseminating this sort of analysis is to remind folks to have humility before the facts. Or, to put it another way, to check their assumptions now and then. Experience can lead to people creating mental short-cuts (rules of thumb) that are easy to apply and generally represent a reasonably good proxy of the truth. The problem is,  when these heuristics either get raised to the level of principles or ossified into dogma.

With rules of thumb, sometimes the nature of the relationship between the variables is lost until all that is recalled is a fuzzy correlation. For instance, good goalies make it much more likely their team will win because they inhibiting the opposition’s ability to score more goals. Therefore, it’s a good heurisitic to say "good goalies have good win totals". On the other hand, goaltenders have no control over how many shots their teams give up or how many goals their skaters score, which are also key factors in winning, making it entirely possible to be a good goalie but have an uninspiring W-L record. 

So the proper expression of this heuristic is: "Good goalies will probably have more wins, but not necessarily." What that so often gets simplified into is: "Good goalies get wins," to the degree that the direction of the relationship is inverted: lots of wins means a goalie is good, which is at times completely false and can lead to poor analysis and bad decision making.

This relates to my recent discussion of size and toughness. All things being equal, it’s usually preferable to have bigger, tougher players. It’s important to keep that qualifier in mind, though, because all things are rarely equal. Bigger = better is another one of those rules of thumb that too often becomes over-simplified and deployed as if it’s a principle rather than a rough guide with caveats.