Asking the Right Questions – Content Over Style

 

 

In previous entries in this series, we’ve discussed how GM’s and coaches can fall victim to certain fallacies that result in bad decision paths. In part one, we looked at mistaking correlation for causation in small sample sizes and fabricating narratives to explain randomness. In part two, the topic was overly weighting the effect of vague, unquantifiable factors in players and teams. 

I want to expand on that latter point this time around to cover how player categorization plus qualitative scouting reports can lead folks sometimes to value style over content.

One of the reasons I began to use advanced stats to evaluate players years ago is I found many of the conventional methods didn’t really tell me anything. Let’s take a look at a couple of standard (but hypothetical) scouting reports that lists all of a guys apparently strengths and weaknesses:

Player X

Player Y

Etc, etc. you get the idea.

There seems to be a lot of information there. Most hockey fans could probably construct a reasonable vision of the guys in question just based on those few bullet points. Naturally, the subsequent evaluation of them as potentially useful players will depend on the evaluators own preferences, perception of team needs, etc.

Here’s the issue: the above is primarily a description of the style of game each guy plays which is not necessarily indicative of how effective he is at providing value on the ice. It’s kind of like asking how fast a car can accelerate from 0-60 and getting a laundry list of it’s various qualities and specifications (such as wheel size, interior space, color, engine size, transmission type) in response. There might be suggestions and clues buried within the descriptors, but the question remains largely unanswered.

Because no one is perfect, positive and negative scouting reports can be constructed about the same guy and be completely accurate. Jay Bouwmeester is big defender with huge, fluid strides who can glide effortlessly around the ice. He can play on both special team units, he makes incisive headman passes and is one of the most durable athletes in the league. He is also far less physical than one would expect of someone who stands 6’4", is sometimes prone to give-aways in his own zone when pressured and has a below average shot from the point.

So…based on those competing descriptions: is Jay Bouwmeester a useful player?

Of course, the answer depends on the proportion to which his strengths stand to his weaknesses; in how the sum is constructed from those disparate parts. Two of the most dominant all-around players in the modern era are Pavel Datsyuk and Nik Lidstrom. Neither guy is overly big or physical or known for blocking shots or dropping the gloves. You cold build an intimidating list of the conventional hockey areas/tools which both guys lack. What they do, though, is essential – drive the play. When Datsyuk and Listrom are on the ice, their team outshoots and outscores the bad guys by a wide margin, which is why Detroit wins so often. 

The Red Wings pair does it with puck skills and smarts. Some guys, though, employ a mix of strength and physicality others have great shots or are blindingly fast. There are various ways to skin the cat and what ultimately matters, of course, is that the cat gets skinned.

Which isn’t to say a players physical tools, assets or shortcomings should be wholly ignored, just that his outcomes are ultimately more important. It’s valuable to know how a guy gets around the ice just as long as that doesn’t obscure where he’s going, if you catch my drift.

One area I think where the focus on player qualities over player outcomes can muddle things is in team building, specifically when it comes to player categorization, ie; "roles" on a club. For example, NHL teams primarily employ bigger, tougher players in their bottom six forward rotation, especially when it comes to the the 4th unit. This is often an area where folks stop asking pertinent questions (can this guy outplay his opponents? Does he drive goal differential?) are instead start focusing on particular qualities (is he big? Is he mean? Can he fight?). This is how a person decides, for instance, to buy out Nigel Dawes after a 15-goal season only to to sign Raitis Ivanans.

Again, this is not to say being big and tough is bad or that tough guys can’t be useful – instead, the issue is that being tough also doesn’t necessarily mean a player has any value. Being tough is merely a potential asset, a tool that has utility only insofar as it helps drive play, goals etc. If a guy is big and can hit but bleeds shots and goals against because he’s completely miserable at everything else, then he is a liability. And deploying liabilities because their scouting report coincides with a conventional role or category is ineffective.

To put it more plainly: toughness for the sake of toughness is dumb. 

This error can be replicated in other ways. For example, a team with a slower, less skilled defender corps might decide it needs more mobility on the back-end and so will pursue players whose scouting reports list those assets. Of course, there are plenty of defenders who can skate but can’t sufficiently advance play. The error is looking at a team in aggregate and deciding it needs more of "positive quality X" and that acquiring a player with "quality X" will be like adding some salt to a bland pot of stew.

Again, it doesn’t matter as much if a guy can skate: it matters if he puts that skill to use in a meaningful fashion. Otherwise he’ll likely be detrimental even if it seems the club could really use some speed. It’s not wrong to desire a mix of qualities or to try to build a team with a balance of abilities. But those concerns should ultimately be subordinate to the question of how effective a guy is at driving possession and goal differential.