Babchuk vs Regehr, Part Two

A lot of people think that statistical analysis of hockey is complete nonsense, and last week we showed how these detractors appear quite justified when virtually every defensive statistic agrees that Anton Babchuk is as good as (or even better than) Robyn Regehr defensively.

We saw how these statistics appear to do a great job in determining the defensive strength of an entire team, but appear to be lousy at assigning those defensive contributions to the individual players.  This week we’ll discuss an alternative way of assigning credit, apply it to the Calgary Flames defense for every season since the lock-out, and see if it passes the sniff test.

Assigning Defensive Contributions to Individuals

Having concluded last week that the 20.3 defensive point shares awarded to the Calgary Flames blue line in 2010-11 looks reasonable, how should it be assigned to the players individually?

• The simplest way is just to divide it up by ice-time, but that would mean that Jay Bouwmeester contributes just as much per minute as T.J. Brodie.
• Instead we could weigh it using something like +/-, but that has obvious flaws. Besides, his +18 score would still make Anton Babchuk look like the 2nd coming of Rod Langway.
• We could incorporate the type of advanced statistics that help put a player’s numbers in context, like Offensive Zone Starts and Quality of Competition, but now we’re getting awfully complex and inaccessible.

So what’s the easy solution? To me the key to assigning individual defensive contributions is based on something simple like ice-time, and avoids fancy statistics like Zone Starts and Quality of Competition, and I think we’ve got it.

The inspiration comes from Points Allocation – a system introduced by Iain Fyffe that actually pre-dates any of the methods discussed last week (GVT, PC , etc). Fyffe’s approach to assigning individual responsibility for a team’s over-all defensive success (or lack thereof) was based on the premise that any ice-time that can’t be explained by a player’s offensive contributions must be explained by their other contributions, the foremost of which is their defense. Brilliant!

An approach that would somehow peel the offensive component out of their ice-time would settle our sample controversy quite admirably. Check out how, at even-strength, Anton Babchuk outscored Robyn Regehr by almost double, 1.07 to 0.57 points per 60 minutes. Despite that clear superiority offensively, Robyn Regehr enjoyed far more ice-time at even-strength: 17.23 minutes per game to 12.98.

The Fix

All we need to do is figure out how much of a player’s ice-time was earned by their non-offensive contributions, and assign the team’s defensive contributions accordingly.

I’m not sure how Iain Fyffe does it, but I figure the most elegant way of approximating it is by adding:
• Their short-handed ice-time (which is defensive only)
• A percentage of their even-strength ice-time equal to the amount of all special teams time they spend short-handed
• Ignore their power play ice-time (which is offensive only).

At first I wrestled with a way to use their even-strength scoring to estimate what percentage of their even-strength ice-time was based on their defensive contributions, but it was complicated, subject to a lot of scoring luck, and wound up being quite similar to a player’s special teams time anyway.

Here’s how it looked for all Flames defensemen in 2010-11.

Much better!  The two systems largely agree, but give defensive-minded blueliners like Robyn Regehr, Cory Sarich and Brendan Mikkelson a bit of a boost at the expense of offensive-minded defensemen like Anton Babchuk and Mark Giordano, while leaving two-way defensemen largely untouched. Unfortunately the new system doesn’t allow for negative defensive value, so last year that helped T.J. Brodie and Ian White.

These final results look surprisingly reasonable given the quick and dirty nature of the approach. Obviously a more sophisticated approach to the even-strength ice-time percentage could fine-tune this further, but at the expense of that additional and often inaccessible complexity.

Looking Back

In our quest to come up with a satisfactory solution to the Anton Babchuk vs Robyn Regehr controversy we may have created a model that doesn’t make sense for the other players and other seasons, so let’s hop in our Delorean and look back at previous years.

Let’s study former Flames, given that we have both the experience to adequately judge their defensive abilities, and yet are a little more objective than we’d be for those still wearing the Flaming C.

As expected, the new approach works very well for the more offensive-minded and/or two-way defensemen like Dion Phaneuf, Roman Hamrlik, Ian White and possibly Adrian Aucoin, too.  In each case their overstated defensive contributions were brought down to a more realistic level, and smoothed out.

Dion Phaneuf

Roman Hamrlik

Ian White

The graphs show Aucoin’s transition into a more defensive-minded blueliner, whose defense has surpassed Roman Hamrlik’s solid two-way play, and makes the distinction between them and more offensive-minded players like Dion Phaneuf and Ian White, whose defensive play is consistently reasonable.

The new approach works a little less well with more defense-oriented players like Robyn Regehr, Steve Staios and Jim Vandermeer, but isn’t really that much worse. Actually, the system really blew it with Steve Staios, but in its defense the Edmonton Oilers had virtually no defensive options those two seasons and gave him a lot of ice-time he probably didn’t earn with his defensive play.

Robyn Regehr

Steve Staios

Jim Vandermeer

Finally there are defensemen like Jordan Leopold and Andrew Ference who are a little tougher to pigeon-hole. Either way the two approaches don’t yield spectacularly different results, and a solid case could be made for either trend line.

Jordan Leopold

Andrew Ference

The Verdict

Evaluating a team’s defense as a whole doesn’t seem to be a major problem, but assigning credit to the individual players obviously has been, which is why we get silly results like Anton Babchuk being equivalent to Robyn Regehr, and the subsequent lack of credibility in the eyes of the common fan.

While we didn’t intend to propose a perfect solution meant to completely restore everyone’s faith in objective statistical analysis, we at least demonstrated that there is clearly a path that heads in the right direction.

Even with a quick-and-dirty approach to figuring out how much of a player’s ice-time is spent playing a defensive role and assigning credit accordingly we demonstrated that you can avoid the silly Babchuk/Regehr results without significantly affecting the more accurate results scored with everybody else.

I look forward to your comments on these results, and if there’s interest I can compute them for the Flames current defenders, too.

• BobB

I think you’d be better off separating yourself from the “point shares” system.

The concept of a players ice-time that is not offence is defence is so fundamentally flawed, that although any results may be encouraging, it’s all going to be meaningless to many people.

There are lots of players who spend a whole chunk of time on the ice doing nothing at all other than skating around. Their “chips” are missing both buckets. Jarome’s defensive “point-chip” is not what Daymond Langkow’s “chip” was.

Even saying a goalie only stops pucks and lets in pucks is closer, and that’s very much over-simplified.

This is my problem with “advanced stats”. Some guy somewhere comes up with a factor, or a “chip” analogy and although it might generally work, we start prescribing it massive value.

“Dude is a 2.5 goals per point chip under the threshold of peanut butter at even strength tied, he IS worse.”

Cory Sarich being a 4.3 vs Gio’s 2.2 is going to sound the same as Babchuk being a 3.5 “point shares” vs Regehr being 3.5.

I think we need to come to conclusions with the least amount of “conversions” involved. The NHL.com #’s lack clarity, so we need more clarity not different “lacking clarity”

• Willi P

Your right RD, but this is why you can never replace people. If there was some way to make it so that we could assign numerical values to everything in the game, anybody could be a GM in the league. That’s why when we play NHL on our Xbox we can make informed decisions with relative easy, numerical values at every spot.

So my point is that stats are at best a basic indicator, there is no replacing opening your eyes and looking at the game and making an informed decision.

The part that is funny is that it is just this fact that makes the backlund situation so polarizing.

• RexLibris

Hi Robert. A nice read. I’m not especially mathematically-inclined, but have enjoyed the comparisons and analysis.

Your observation of Staios is appropriate and criticism of his play during those years focused on the fact you mention, that he was essentially placed far too high in the batting order for his skills. The same circumstance should be applied to Vandermeer’s stats in 2010 when he was on the Oilers. The team that year was hemorrhaging goals, and even though he was a decent option for defense, nobody was going to survive that season.

One thing that stood out for me while reading this was that, I have often advocated for a balanced defensive core, without a single cap-dominating defenseman. Rather than having a Pronger-esque defenseman followed by a cohort of 3s and 4s, I prefer the model of having a good first and second pairing followed by defensemen at 3 through 5 with little differentiation, while the 6th spot is ideal for developing defensive prospects and rotating in more veteran defensive depth when appropriate. To my mind this prevents the team from locking up too many resources in a single asset and insulates a roster against serious injury. To this end, while I would like to see Ryan Suter sign in Edmonton this summer, I strongly believe that it would have to be a cap hit in the range of \$5 million a year. Having a defenceman like Shea Weber take up \$7.5 million of a team’s salary-cap space is simply too crippling to provide the balance needed to be successful.
As much as I dislike the team, Vancouver’s defence is structured this way and has proven fairly resilient in their playoff runs (Sami Salo aside). As I recall, during the Flames’ peak this was perhaps also one of their strengths, but then I could be mistaken.

In forecasting the potential defensive situation in Edmonton two to three years from now it is my belief that this is something the management here is planning for. Petry is on pace to be a replacement-level defenseman for Gilbert, while out of a crop of Simpson, Musil, Klefbom, Marincin, Gernat, Plante, Teubert, Fedun, Bigos, Blain and Davidson the odds (and scouting reports thus far) suggest that there are at least three good NHL defencemen in that group, perhaps more.

In Calgary’s situation, with so many UFAs coming off the books (presumably) and at long last an opportunity to promote some forward positions from within, there is a real opportunity to bolster a defensive core of Bouwmeester, Brodie, Babchuk, Butler and Giordano with a player like Brad Stuart, Willie Mitchell, Johnny Boychuk, or even a Ryan Suter. A six-man rotation with those players ought to provide enough defensive consistency and forward support to help improve the team.

Anyway, sorry if I hijacked the article. I just thought the topic lent itself to this sort of discussion.

• Robert Vollman

Thanks for the comments so far, they have been quite fair and reasonable.

I believe for years that advanced statistical analysis in hockey has been ignored.

Despite how incredibly useful it has already proven there is a resistance to it, apparently because it can only explain some things – not everything – and the fashion that it explains those things aren’t always clear.

For example, Minnesota fans may acknowledge that advanced statistics forecast their drop in the standings, but since they don’t really understand how or why they still see no value in them.

I truly believe statistics can help you both understand and enjoy the sport we already love so much, and I’d like to find ways to bridge the gap.

To many it may seem like too big a job, but you know what they say about eating an elephant. Anyone can eat an elephant if you do it one bite at a time. And I’d add the following caveat: don’t start with the ass.

I’m glad to have taken one of the easier nibbles: Regehr’s better defensively than Babchuk.

• RexLibris

*furiously taking notes, writes* “don’t….start….ass…”. Got it!

• RexLibris

Rob,

Really interesting work.

My only concern is that the basic allocation is based off of ice time. Ice time is basically governed solely by the coach’s decision. While I would agree that most coaches, most often get the players most likely to give them the best chance to win out for the most ice time, it is not absolutely the case. In fact, I have always thought that at least one of the goals or objectives of looking at advanced stats was to reveal situations in which a coaches use of players might be suboptimal.

I think this methodology is fair in the way it would create rankings, but will be flawed in never revealing a player who might be able to make a greater defensive contribution but the coach does not perceive the contribution and does not play that player in a large number of minutes.

• Robert Vollman

I agree, there are definitely pitfalls in using ice-time in this fashion.

I didn’t intend to say “Here: this is the solution” (to how to allocate individual credit for team defense) but rather demonstrate that a solution exists.

• ChinookArchYYC

“I have always thought that at least one of the goals or objectives of looking at advanced stats was to reveal situations in which a coaches use of players might be suboptimal.”

My thoughts exactly.

• BobB

Confirmation Bias?

I’m guilty as well as I think your numbers do look better, but the bias is definitely lurking.