Random Thoughts – On Process vs Results



The recent struggles of Sean Monahan is a topic of interest for me, in part because it sparked a discussion about the nature of analysis, particularly regarding valuing results and processes. Let’s get to the Monahan stuff first and the rest will follow. I’ll talk about Mikael Backlund (yes, again) here as well.

Monahan’s struggles

Since returning from injury, the Flames rookie center has been getting absolutely killed. Not including last night, Monahan had been outshot (including blocks and missed shots) 48 to 15 or -33 over just three games at even strength. That’s a corsi % of 24%(!!). To put that ratio in context, the worst regular skater in the league by this measure so far this year is the Leafs Frazer McLaren at 36.6%. Brian McGrattan’s corsi% is 41.1% so far (worst forward on the Flames).

Keep in mind, this is while playing with Curtis Glencross and Jiri Hudler and getting fairly generous zone start bumps – the kid’s offensive zone to defensive zone faceoff ratio in those three games was 80%, 55.6% and 60%, respectively. So circumstances don’t explain him getting run over.

Even in such a small sample of games, these are worrying possession numbers and suggest something has gone terribly wrong. Either Monahan has hit the dreaded "rookie wall" or he has come back from injury too early. Either way, his night in Detroit began the same way, if you’re wondering why the kid was benched in the third period.

Anyways, my noting the Monahan’s dreadful outshooting numbers after the Rangers game (22 shots against, 6 for, -16 corsi) led to some discussion on Twitter. I was reminded that the Monahan line scored two goals despite being grossly outshot and that those are the only results that matter. Two goals for and zero goals against means a good night of work, all other considerations be damned.

Well, not really. As I note here continually, the best long-term predictor we have for outscoring in the NHL is outshooting (ie; positive possession). I think even the most innumerate amongst us can also recognize that getting outshot 22-6 but coming out with a 2-0 goal differential is more horseshoes than ability.

This brings me to the discussion of results vs process, which I’ll illustrate with a personal anecdote:

Years ago some friends and I used to get together monthly for a night of texas hold’em poker. This particular evening I was fortunate to be one of the last two players at the table. The game had gone on for a long time, it was a weeknight and my stack was beginning to dwindle. We also gave out a portion of the pot to the second placed guy, so with the guarantee of winnings I simply decided to throw the game and go home.

On the next hand, my two hold cards were a 2 and 9, off-suit. Perfect, I went all-in before the flop. My adversary called with something much stronger – a couple of face cards or something similar. He was much more likely to win, which was my intent anyways.

Except the flop happened to come up 9, 2 other. I had two pair and I went on to take the hand. I got a good result despite a terrible, low percentage play. It happens in cards.

And it happens in hockey. One of the reasons I was drawn to better, more predictive metrics in the early days of my own hockey writing is because I found myself too often chasing my tail in terms of my assessments of teams and players. I frequently found that the "numbers that matter" – ie the results like goals and wins – didn’t seem to accurately predict who was actually the better player or club down the road. I found myself bouncing around and flip-flopping, resigned to deploying shopworn, retroactive narratives like confidence and other pop psychology to explain away things when they didn’t accord with what had seemed to be solid conclusions.

Like a bad poker player who chases flushes, projects his betting habits and doesn’t understand how to bluff, bad hockey teams and players can nevertheless experience pockets of success. Sometimes you get the cards. Sometimes the puck bounces off a skate and goes in or your goalie stands on his head. Sometimes luck can favor you for longer than a game or hand. But, eventually, playing lousy catches up to you.

The objective in hockey is to score more goals than the other guy. When that happens, it can be seductive to assume that the game was played well. But it can be a false signal. Process matters. That’s why, despite winning big with a 9-2 off suit that one time, I still fold the hand whenever it pops up.

Backlund’s Ascension

Which brings us to Mikael Backlund, whose recent spate of good play has ironically made him a player of contention in some quarters again.

Let’s first break down his last few games like we did with Monahan – in the last three, Backlund’s raw possession numbers have been +33 -28 for +4 or a ratio of 54%. This is especially noteworthy because the Flames have been soundly outshot in each of those contests – Backlund’s corsi rate relative to the rest of the team in those games was +15.7%, +32.9% and +13.3%. Unlike Monahan, Backlund wasn’t starting more shifts in the offensive zone either – his o-zone to d-zone faceoff rate was 22%, 23% and 45%, respectively.

So that’s some quality work and the reason why Mickis has lead the Flames forwards in ice time recently. You have to go back to the end of November to find the last time Backlund was scored on at even strength (against the Ducks), which is a function of good play and luck (100% ES SV%!). Still, this is Backlund’s under appreciated talent – the puck moves the right direction when he’s on the ice. He doesn’t do it by hammering opponents with big hits or making Datsyukian dekes through the opposition. It’s just quiet, dogged puck pursuit and an ability to read and react to the play. Which is why he’s an uninspiring player to many pundits and fans – we tend to notice and remember the profane and spectacular and not much else.

One of those pundits is Dean Molberg (Boomer) on the FAN960 in town. A recent exchange between him and friend of the program Ryan Pinder on the radio this past week sparked me to write this section…

The argument was over Backlund, with Pinder’s defense couched in possession metrics. Boomer, a staunch disbeliever in corsi, stated that whatever the numbers say, Backlund seems too passive and uninvolved to be a player of any value. The argument devolved into Molberg more or less telling Pinder to step away from the "nerd stats" and watch the game with the implication being that Backlund would fail the eye-test absent the crutch of arcane measures. 

Let’s first establish that the legitimacy or utility of evidence is not subject to any given person’s subjective appraisal or incredulity. Information is information, even if what it implies is personally hard to digest or fathom. Every major discovery or advancement of knowledge in human history was initally met with disdain, contempt and dismissal by people in general and established authorities in particular. Truth is immune to popularity.

Furthermore, let’s also establish that anyone who has to muster a smear or stigma to invalidate evidence is immediately behind in an argument. One of the easiest ways to rationalize away uncomfortable or counter-intuitive information is to label it as somehow heretical and therefore not worthy of serious contemplation.

An analogy…the theory of evolution is based on lots and lots of data and evidence. It’s the best understanding we have of how life grows, changes and persists on this planet. But there is a large swath of so-called "creationists" who dismiss evolution out of hand because its conclusions do not accord with religious dictum. So the evidence behind evolution is labeled blasphemous and redacted from the minds of true believers because it is rationalized as inherently wrong. 

My intent is not to call Dean the hockey version of a creationist (seriously…I’ve met Boomer and like him), but to point out how  cognitive dissonance can make people act and think in less than rational ways.

Check Your Premises

The irony is that suspicion of numbers is not unwarranted. As I mentioned above, people most certainly should be suspicious of the value of outcomes like wins, goals, points, etc. (particularly in small samples). One wonders if the established hockey orthodoxy grew distrustful of stats over time because so many of them (*ahem*plus/minus) didn’t paint an accurate picture of a team or player.

The problem is, no one treats their observations with a similar level of suspision. We’re hardwired to believe what we see and, further, to take subjective impressions and compelling anecdotes as implacable indicators of truth and reality. No matter how much you respect the power of a given metric or series of numbers, they never seem as viscerally real as the instant judgements and gut feel we get from watching tha action.

Which is why it’s sensible to respect and investigate quantitative evidence and admit, at times, that subjective evaluation might not completely line-up with reality. Humans are good at naturally detecting patterns in data, but we’re terrible at telling when those patterns represent real causal relationships or not.

I’ll finish by noting that the current disruptive "new stats" like corsi and PDO that give so many established MSMers (and some authorities in the league) fits weren’t constructed by theorists in a lab far away from the game. They were stumbled upon and tested in a series of iterative studies by passionate, invested hockey fans and analysts – a sort of crowd-sourced empiricism where the utility of possession metrics and percentages was tested and re-tested and then built upon when they proved to be relatively robust. To put it another way, these aren’t numbers cooked up by people who don’t watch hockey and are simply looking to completely up-end current conventions because "screw you jocks!"

For my part, I think the point of discussing and disseminating this sort of analysis is to remind folks to have humility before the facts. Or, to put it another way, to check their assumptions now and then. Experience can lead to people creating mental short-cuts (rules of thumb) that are easy to apply and generally represent a reasonably good proxy of the truth. The problem is,  when these heuristics either get raised to the level of principles or ossified into dogma.

With rules of thumb, sometimes the nature of the relationship between the variables is lost until all that is recalled is a fuzzy correlation. For instance, good goalies make it much more likely their team will win because they inhibiting the opposition’s ability to score more goals. Therefore, it’s a good heurisitic to say "good goalies have good win totals". On the other hand, goaltenders have no control over how many shots their teams give up or how many goals their skaters score, which are also key factors in winning, making it entirely possible to be a good goalie but have an uninspiring W-L record. 

So the proper expression of this heuristic is: "Good goalies will probably have more wins, but not necessarily." What that so often gets simplified into is: "Good goalies get wins," to the degree that the direction of the relationship is inverted: lots of wins means a goalie is good, which is at times completely false and can lead to poor analysis and bad decision making.

This relates to my recent discussion of size and toughness. All things being equal, it’s usually preferable to have bigger, tougher players. It’s important to keep that qualifier in mind, though, because all things are rarely equal. Bigger = better is another one of those rules of thumb that too often becomes over-simplified and deployed as if it’s a principle rather than a rough guide with caveats.

  • Lordmork

    I basically never listen to FAN960, but I happened to catch that exchange because I’d been listening to the game on the way home the night before, and it struck me that Boomer’s defence was basically that Backlund fails “the eye test.”

    Unlike a lot of people on this site, I’m really not a hockey guy. Never played the game, never cared about it as a kid, only started getting into it over the last few years. So I’d be the first person to admit that I don’t have the eye for the sport that some people do.

    So I find this conversation really interesting. I really like Backlund, both for what he brings to the team, and off the ice, and I find that what some people criticize him for are things that are basically invisible to me. What he does well is quite often also invisible to me, but because of corsi and possession rates and such, I know he does them.

    Generally, I just like watching him play. I hope he’s able to elevate his play another notch so these questions about him are finally quieted, and that he can secure his position on this team.

  • mattyc

    “The problem is, no one treats their observations with a similar level of suspision. We’re hardwired to believe what we see and, further, to take subjective impressions and compelling anecdotes as implacable indicators of truth and reality.”

    A good example of this is my opinion of Oli Maatta. Over the course of the 2011/2012 season, I had heard lots of good things about him and seen that he was likely to go right around where the Flames were probably going to draft (you know, just out of the playoffs). So naturally, I got intrigued. Then I watched him a few times, and came away completely unimpressed–to the point where I actually had no interest in him at 21st overall (I was hoping the Flames would take Sebastian Collberg). Now, Maatta is filling in on the Pens’ top defense pair, and starting to garner some Calder attention.

  • piscera.infada

    Whoa, whoa, whoa! So you’re saying there’s no skill in horseshoes!?

    Great article, much appreciated! I do however, somewhat agree on the Boomer-Backlund thing. I get the Corsi numbers, and I agree the guy’s a player. But I also, by nothing more than the eyes-test, do tend to agree that his issue is consistency. When the guy’s on, he’s great – there’s no arguing that. He’s yet to “bring it” game-in, game-out – and I do think Hartley’s handling of him is somewhat in line with this. That’s not to say Hartley’s handling of Mickis is necessarily right in terms of icetime, but I think Backs is at a point in his career where consistency needs to be a point of emphasis, or else it’s going to be tough for him moving forward.

    • This however is the problem. The eye test is subject to biases. You only remember what stands out in your mind. If an airplane crashes people think Airplanes are readily unsafe because an airplane crashing is so infrequent it stands out in your mind. However, numbers wise airplanes are the safest form of travel even ahead of walking. The regularity of travel by car leads us to think cars are safe when in reality by numbers they’re the worst form of travel. The eye test fails. One particularly bad game Backlund has is always going to be the memory that comes to the forefront and I suspect the Backlund detractors remember one particular moment. And the point Kent makes is Backlund is not the type of player who makes big plays that stick around in your memory. However, the small constant things he does do indicate that over time he will be more consistent and more valuable to a team that’s rebuilding.

      Keep in mind that the first 3 years of Backlund’s career have been mired by bad starts and injuries at untimely moments. He’s finally had a season so far where he’s started from the beginning. He struggled at first even though his underlying numbers were good. But those underlying numbers predicted this success later on. And now here we are heading into the mid season and Backlund is the most consistent Flame who provides them with the best opportunity to score points on a consistent night-to-night basis. Another example, the Leafs were getting it done early in the season. But people who read the underlying numbers predicted this substantial fall in success. The relatively Flashy and stand out play of Stajan has led some people in media by the eye test to believe that the Flames should resign Stajan to a contract to insulate the young players. The problem is look at his stats longterm. Sacrificing Backlund to keep a 30 year old because he’s more flashy is suicide. The eye test would have had everyone write Backlund off at the start of the year and they did. The objective test of the numbers however had people who knew how to read them saying stick with the kid give him a chance he’ll prove to you over a larger sample of games that he is worth it. In my opinion, we’re just beginning to tap into Backlunds ability and he’s 25 and the healthiest he has been.

      • “One particularly bad game Backlund has is always going to be the memory that comes to the forefront and I suspect the Backlund detractors remember one particular moment. And the point Kent makes is Backlund is not the type of player who makes big plays that stick around in your memory. However, the small constant things he does do indicate that over time he will be more consistent and more valuable to a team that’s rebuilding. ”

        I’ve seen something really similar happening with TJ Brodie recently, actually. I have seen fans online who would trade Brodie for Jake Gardiner straight up without even thinking about it.

      • piscera.infada

        Read above, I’m not biased against Backlund, and I’ve also been watching/playing enough hockey throughout my entire life that I feel I can get an fairly decent grasp on what is going on with players/teams. Sorry I responded with so little effort to what was a lengthy post, my boss is looking at me with that “weird, slack-off-ish look”.

        Also re: defensive centres, my favorite player in the league is Bergeron. So, there’s that.

    • Parallex

      I don’t see any issue with consistency. There are zero players in the NHL that are perfectly consistent in all facets of the game and Mickis is no exception, his “consistency” is fine. Even when he’s in a scoring slump he still brings his dogged pursuit of the puck (Aside: this is my favorite part of his game, Backlund forces the oppositions hand better and faster then any other player on the Flames I’ve lost count of the number of times he forces them to lose possession of the puck simply by limiting the amount of time they have to develop a play), solid defensive play, and up ice movement. Honestly I think he’s had only 1 bad game all year (Winnipeg).

  • prendrefeu

    Great write up, Kent. Thanks.

    Is there any chance Flames administration (read: Burke, King, Hartley, et al:.) read this article? Do they read FlamesNation?