Mark Seidel on the Art of Scouting

Kent Wilson
May 19 2010 09:04PM

NHL Draft Lottery Drawing

 

I stumbled across a very interesting article by the Chief North American Central Scout Mark Seidel today. Entitled "Why NHL Teams Fail at the Draft" much of what he discusses resonates with my own thoughts on prospect evaluation. It's also relevant for the Calgary Flames, a team that clearly fails at the draft

1. Overvaluing performances in high-profile games

Certain tournaments such as the world junior championship, the Memorial Cup and specialty "all-star games" like the CHL Top Prospects Game contribute greatly to the mistakes that NHL teams make — especially when it comes to first- and second-round picks.

The reason is that these high-profile events tend to attract NHL generals managers and presidents who, by watching just a small sample of a player's performance, can fall in or out of love with a prospect.

...

For example, I know of one team that had its GM go to the Prospects Game and absolutely fall in love with a player that the whole staff had agreed wasn't worthy of being a first-round pick. His off-ice character was sketchy, he was a bad teammate, and everybody knew he was a very risky pick. The scouting staff was in its first season with the team, and the more the GM raved about the prospect, the quieter they grew because they didn't want to disagree with their boss.

Even the head scout started to hedge his bets. Eventually, he started to try and find positive things to say about the prospect and, on draft day, the team found the player still available at their pick in the 20s. They practically ran to the podium to scoop him up. The prospect never panned out, and the team wasted a valuable first-round pick.

His first point is perhaps the most instructive, for two reasons: the first, that sample size matters. It's something I emphasize in my own numbers-based thoughts on player performance, but it's just as important on the qualitative side of things. Player performance is highly variable and the bell of the ball one night can be the ugly sister the next. Only over repeated viewings can one get a true sense of a players mean (or true) level of ability. 

The second is the description of how decision making can be skewed in rigidly hierarchical organizations like a hockey organization. Seidel's anecdote sounds suspiciously like groupthink...a type of thought within a deeply cohesive in-group whose members try to minimize conflict and reach consensus without critically testing, analyzing, and evaluating ideas.

Some symptoms of groupthink include rationalizing away conflicting information, excessive optimism and self-censorship by group members. It is especially prevalent in top-down directed organizations in which there is a homogeneity of social background and ideology amongst the members. It is at this point that I should point out that the Flames could probably change their name to the Sutters (and their logo to an image of Darryl's scowling face) and it wouldn't be completely unexpected.

Of course, I can't know that groupthink has influenced the Flames own shoddy drafting under the Sutter regime: hell, the team was terrible at it before he took office. However, with a monolithic "Sutter culture" apparently fully entrenched in both the brains and bowels of the organization, I can safely say that it's certainly possible. 

2. The 'big fish, small pond' effect

Another important situation for teams to avoid is falling in love with a prospect who is playing against inferior competition. This can be true for kids playing high school or Tier II junior because a prospect that is just OK can look like Wayne Gretzky against weak opponents.

This is another point I try to make when looking at prospects, especially about kids playing in highschool (they play about 25 games a year) and lesser leagues like the BCHL. It's especially problematic if the guy is playing on a line with an even more dominating teammate

Seidel's other two red flags are "poor character" and "ignoring age effects". The former I can't speak to because I don't interview players, but the latter is something easily missed at the junior level and especially in kids playing in college. You'll notice a lot of highly ranked guys don't boast impressive totals in the NCAA. Usually that's because college hockey has a lot of older guys (20 and above), making it much more difficult for teens to compete or even be granted much ice time as freshmen. It's therefore very important to read a player's results in context of his age cohort. 

Anyways, these are just the broad strokes of the article and I recommend reading it through fully. 

39d8109299a9795cb3b41a4e9b49d501
Former Nations Overlord. Current FN contributor and curmudgeon For questions, complaints, criticisms, etc contact Kent @ kent.wilson@gmail. Follow him on Twitter here.
Avatar
#1 mclea
May 19 2010, 09:42PM
Trash it!
0
trashes
Props
0
props

Good read.

The second is the description of how decision making can be skewed in rigidly hierarchical organizations like a hockey organization.

This is an interesting idea. In order to have a successful scouting departing, I believe you need to have the following:

1) A staff that is properly valuing players. You have to know what skills in junior hockey are most likely to translate to the NHL and what their relative value is. You have to correctly asses the risk that a particular player will not develop as expected. This is the kind of stuff you stat nerds will some day generate that will complement the visual information compiled by dozens of sophisticated scouts. You will generate information that will help analyze past draft results, and use this information to help make better decisions going forward.

2) You must give the guys with the best information, and the skill sets above, decision making powers.

The Flames obviously have major issues on the first point. They did not adapt to the fundamental changes in the way the game was played following the lockout. Sutter was good at identifying talent for the old NHL, but he was unwilling to change the way he thought about the game because the existing paradigm he worked under gave him so much success. The game passed him by, and his past success kept him from catching up.

We're seeing something similar now with goaltenders. The equipment change has not only decreased the value of a "good" goalie vis-a-vis other goalies and other positions, but it has also changed the way a goaltender performs their job function, which has resulted in a change in the optimal skill set for the position. A goalie that was successful with huge equipment is not necessarily successful with smaller equipment. We saw something similar in the NBA with the hand check rule, and the NFL with the stricter illegal contact rule for DBs. The way the game was played changed, and consequently so did the way you valued players.

In the anecdote, the team had problems with the second point. They had good information with the right people evaluating it, but these people were overruled by the GM. There is no point setting up an expensive scouting system if you're just going to ignore the information that is created from it. If you can't trust your scouting staff, why do you have it?

Avatar
#2 Grant F
May 19 2010, 09:43PM
Trash it!
0
trashes
Props
0
props

It's an interesting article for sure...however I think it is too difficult to simplify bad drafting down to a list like this.

For instance, one could make a good argument that the Flames in fact are guilty of over-valuing character, drafting guys that are short on skill but big on heart. And heart only gets you so far.

The way I've always looked at it is as you move along in the draft you have to make decisions about what you will settle on. At the high end of the first round the kids have it all - size, skill, character, skating, character - the entire ball of wax.

As you move deeper into the 1st and beyond you start losing some of those attributes.

Do you draft the short guy with a ton of skill, or the rough n' tumble large dude that doesn't skate well?

Do you take the winger with the great shot but lacks hockey sense or the big smooth centre with off-ice problems?

Hard choices have to be made, and I don't think you can break it down to a list of "common errors" like the author is trying to do.

My 2.5 cents.

Avatar
#4 Hayley Mutch
May 19 2010, 11:18PM
Trash it!
0
trashes
Props
0
props

I read this article last night and was trying to think of how to incorporate it into a post. Nice work as always.

Avatar
#6 maimster
May 20 2010, 12:09PM
Trash it!
0
trashes
Props
0
props

Re: character of the players having a major effect on their chances of making it.

I recall back in the late 80's early 90's, when I was getting the Hockey News I'd always look forward to the Draft Preview issue, and I kept them for years afterwards to go back and see what was written about the high-draft players at the time.

And I don't exagerate in saying that over a 6-7 year stretch, EVERY player that was a projected first rounder that had a comment like "scouts wish he'd apply himself a little harder" or "doesn't bring the full effort every night" or "some questions about their focus" eventually failed. Without exception.

Granted, that was years ago, but I doubt things have changed much. You don't need to draft for 'heart', but you definitely need to avoid the guys who don't show their all before being drafted.

Avatar
#8 Middleborn
May 20 2010, 03:19PM
Trash it!
0
trashes
Props
0
props

I can't remember but was Todd Button held over from the Craig Button days of Calgary? I agree that Daz has a good eye for talent, but I also agree that it's the "old NHL" style player he can easily identify. Are the current crop of scouts for the Flames "old NHL" or "adaptable" scouts?

Recently, with the discussions of the issues the Flames organization will be dealing with in future drafts, I've been thinking of Mickey Renaud, the Flames prospect who suddenly passed away at home. Would our worries be as dire if Mickey had not passed on? I have been thinking of "what if" but then it all doesn't matter really; it is unfortunate he was taken so early, but what's done is done. He was a 5th rounder but was Captain of the team, a team that is doing very well.

Ultimately, I think there is an immeasurable quality that some people have that allow them to be successful. But for the most part, gut feeling should be used after the numbers have been crunched.

Avatar
#10 Grant F
May 21 2010, 08:56AM
Trash it!
0
trashes
Props
0
props

A related thought - I think NHL teams get too much blame and perhaps credit for their picks.

What about the player?

These guys are drafted because they have shown a certain set of skills, and have specific physical tools.

Often the reason they don't make it - isn't because they were a bad pick - but rather because they aren't willing to put in the work, or adjust their game to the NHL style.

My point is - we wag our fingers at the NHL organizations and say 'player x was a bust' when the reality is that the biggest reason why that player turn out - is because they didn't hold up their end of the bargain.

Comments are closed for this article.